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1.  SUMMARY 


The  human  brain  is  the  most  powerful  adaptive  network  known  to  man.  It  is 
responsible  for  human  intelligence  with  operations  involving  automated  image 
recognition,  speech,  decision  making  and  complex  motor  functions.  The  same 
functions  are  the  goal  of  artificial  intelligence  operations,  robotics  and  the 
like.  '  Attempts  to  design  machines  to  perform  these  functions  successfully  would 
benefit  from  an  understanding  of  how  the  brain  has  succeeded  in  doing  so. 

Both  brain  and  siachine  depend  on  component  operations.  In  the  brain  the 
basic  component  is  the  neuron.  Although  little,  as  yet,  is  known  of  the  rules 
by  idiich  adaptive  neural  changes  are  brought  about,  experimental  studies  over 
the  past  15  years  have  uncovered  direct  evidence  that  such  changes  occtir  and  can 
be  studied  at  the  level  of  single  nerve  cells  in  the  mammalian  brain  and  in 
sisipler  invertebrate  systems  (Alkon,  1979,  1980a, b,  1982;  Carew  et  al.,  1983; 
Kandel,  1976;  Randel  and  Spencer,  1968;  Woody,  1974,  1977b,  1982a, b;  Woody  and 
Black-Cleworth,  1973). 

Our  studies  have  shown  that  single  cortical  neurons  adapt  in  such  -a  way  as 
to  support  learned  behavior.  What  is  particularly  interesting  to  us  is  the 
indication  that  purposefully  complex,  "lock  and  key"  molecular  cascades  exist  at 
the  level  of  single  nerve  cells  to  permit  "successful"  adaptations  to  occur. 
"Successful"  adaptations  are  defined  as:  (a)  producing  the  desired  alteration 
of  response  to  the  appropriate  input,  (b)  enduring  over  time,  (c)  not 
interfering  with  other  adaptations  occurring  for  other  piurposes  in  the  same 
cell,  and  (d)  not  interfering  with  the  main  -  throughput  -  message  transfer 
property  of  the  nerve  cell.  The  result  of  these  adaptations  is  to  support  the 
operation  of  a  self-organizing  infonsation  processing  system  with  a  high 
success:  error  ratio  and  excellent  survivability  in  the  face  of  substantial 
environmental  change. 

The  nerve  network  and  elements  that  were  studied  reflect  a  much  different 
design  from  that  found  in  single  elements  of  most  artificial  information 
processing  devices.  Nonetheless,  the  design  semns  understandable  in  terms  of 
conventional  information  processing  theory  and  in  terms  of  conventional  systems 
analysis  approaches.  (For  a  brief  summary  of  the  latter  see  the  enclosed 
Appendix  excerpted  from  a  recent  book  of  mine.) 

Some  of  the  complexities  of  these  nerve  cell  adaptive  operations  are 
exemplified  by: 

(a)  A  conductance  decrease  IPSP  that  we  have  found  after  PT 
stiiBulation  (this  mechanism  for  altering  neural  excitability  works  in  just 
the  opposite  way  from  classical  conductance  increase  mechanisms  of  PSP 
generation) , 

(b)  The  fact  that  both  a  strong  depolarization  signal  and  the 
availability  of  a  calcium-activated  protein  kinase  are  needed  to  initiate 
membrane  responses  to  the  latter  agent  measured  in  cortical  neurons. 

We  think  that  we  are  now  catching  "glimpses"  of  some  surprisingly 
sophisticated  ways  in  which  cortical  neurons  adapt  to  support  a  whole  range  of 
newly  learned  tasks,  accomplishing  the  task  at  hand  while  maintaining  the 
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potential  for  acconplishing  others.  Changes  in  the  excitability  of  cortical 
neurons  occur  that  lead  to  acquisition  of  the  ability  to  perform  specific  motor 
tasks  in  response  to  specific  auditory  stimuli.  Rates  of  acquiring  this  ability 
can  be  substantially  increased  by  adding  electrical  stimulation  of  the 
hypothalamus,  associatively,  to  presentations  of  conventional  conditioned  and 
unconditioned  stisuli.  Part  of  this  acceleration  of  learning  the  motor  response 
may  derive  from  recruitment  of  a  new  performance  pathway  -  reflected  in  a  longer 
transmission  latency  for  movement  production.  A  long  range  goal  of  our  research 
is  to  understand  how  the  system  picks  the  "right"  pathway  to  give  both 
acceleration  and  the  "appropriate"  learned  movement. 
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II.  STATEMEHT  OF  WORK 


Postsynaptic  neural  responses  to  stimulation  of  the  pyramidal  tract  and  the 
hypothalamus  were  studied  in  conjunction  with  responses  to  putative  transmitters 
and  second  messengers  thought  to  contribute  to  development  of  conditioned 
behavior  and  acceleration  of  the  rate  of  acquisition  of  conditioned  behavior. 
Details  are  given  in  the  progress  report  that  follows. 

Sixteen  publications  have  resulted  from  research  in  the  two  year  period  of 
this  contract;  see  list  of  publications  -  Part  IV. 
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III.  COMPRmgHRIVE  PROGRESS  REPORT  -  STATDS  OF  RESEARCH 

May  1983  -  May  1985 

1.  The  rate  of  learning  a  conditioned  facial  movement  was  greatly  accel¬ 
erated  by  adding  electrical  of  the  hypothalamic  region  of  the 

brain  to  preaentations  of  conventional  conditioned  and  unconditioned  stimuli. 
Animals  learned  the  OR  after  as  few  as  ten  instead  of  1,000  or  more  pairings. 

The  learning  that  resulted  was  both  associative  and  discriminative  (Kim,  Woody 
and  Berthier,  J[.  Heuronhysiol. .  1983).  That  is,  learning  was  induced  by  a 
specific  stimulus  combination,  the  code  depending  on  the  order  and  interval  of 
presentation  of  the  involved  stimuli.  The  learned  response  was  then  elicitable 
by  a  specific  input  signal.  The  pattern  of  cortical  neuronal  activity  produced 
by  hypothalamic  stimulation  was  predictive  of  loci  of  hypothalamic  stimulation 
that,  when  stimulated,  would  succeed  in  accelerating  learning  (Woody,  Kim,  and 
Berthier,  2.*  Heuronhvsiol. .  1983).  Further  details  are  available  below  (p.  18 
ff.).  Part  of  the  acceleration  of  learning  the  motor  response  may  derive  from 
recruitment  of  a  new  performance  pathway  -  reflected  by  a  longer  transmission 
latency  for  movement  production.  If  so,  one  would  like  to  know  how  the-  system 
picks  the  "right”  pat^ay  to  give  both  acceleration  and  the  "appropriate" 
learned  siovement.  We  would  also  like  to  know  whether  the  hypothalamic 
stimulation  responsible  for  acceleration  of  learning  is  punishing  or  rewarding. 
This  isay,  however,  be  of  less  consequence  in  understanding  what  is  going  on  than 
would  specifying  the  coded  molecular  interactions  that  occur  between  the 
chemical(s)  released  by  hypothalamic  stimulation  and  other  chemicals  capable  of 
modifying  the  transfer  properties  of  the  nerve  cells.  It  is  these  interactions 
that  we  think  are  primary  in  controlling  the  potentiation  of  conditioning  that 
we  have  observed. 

2.  Effects  of  L-ciiitaiiiate  and  L-Glutamic  Acid  Dietbvl  Ester  (GDEE)  were 
studied  in  neurons  of  the  motor  cortex  that  responded  with  short-latencv 
discharge  to  hypothalamic  stimulation.  (Short- latency  activation  of  these 
neurons  by  hypothalamic  stimulation  was  predictive  of  loci  of  stimulation  within 
the  hypothalamus  that  could  accelerate  rates  of  acquisition  of  the  facial  CRs 
described  above.)  Extracellular  recordings  of  unit  activity  were  made  from 
cells  of  the  motor  cortex  of  awake  cats  during  extracellular  iontophoresis  of  1 
M  monosodium  glutasiate.  ^renty  four  of  the  cells  (86Z)  showed  an  increased 
firing  rate  in  response  to  application  of  glutamate.  Eight  cells  that 
demonstrated  short- latency  (less  than  20  msec)  activation  in  response  to 
hypothalamic  electrical  stimulation  were  tested  with  iontophoretic  application 
of  glutamate.  Seven  of  these  cells  showed  an  increased  firing  rate  in  response 
to  glutamate.  Six  of  these  cells  were  then  tested  with  hypothalamic  stimulation 
after  extracellular  iontophoretic  application  of  0.5  M  GDEE  (a  glutamate 
blocker).  The  short- latency  response  to  hypothalamic  stimulation  was  suppressed 
in  four  and  reduced  in  one  of  these  cells.  These  findings,  though  preliminary 
in  nwnber,  suggest  that  some  cells  of  the  motor  cortex  that  respond  to 
hypothalamic  stisnilation  can  be  activated  by  glutamate  and  that  the 

short- latency  response  to  hypothalamic  stimulation  can  be  suppressed  by 
extracellular  application  of  a  glutamate  blocker.  Glutamate  or  some  of  its 
chemical  analogues  such  as  N-acetyl-aspartyl-glutamate  or  N-methyl-D-aspartate 
may  be  involved  in  the  mechanism  supporting  accelerated  rates  of  conditioning. 
(Cooper  and  Woody,  Soc.  Weurosci.  Abstr. .  9:330,  1983) 
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3.  Because  of  the  above  findings »  atteapts  were  wade  to  condition  increased 
activity  to  click  in  single  cortical  neurons  of  awake  cats  using  glabella  tap 
and  iontophoreticallv  applied  glutamate.  (Woody,  Oomura,  et  al,  Soc.  Neurosci. 
Abst..  1984.)  Averages  of  click-evoked  activity  were  compiled  from  single  units 
(n  ■■  104)  of  the  motor  cortex  before  and  after  ten  serial  presentations  of  click 
plus  glutamate  (c  +  g),  click  plus  glabella  tap  plus  glutamate  (c  t  +  g), 
glutasiate  plus  click  plus  glabella  tap  (g  +  c  -*■  t)  and,  in  other  cells  (n  ■  20), 

after  click  plus  glabella  tap  plus  chloride  (c  -f  t  Cl~).  Unit  activity  was  _ 

recorded  through  the  same  electrodes  used  to  apply  0.5  M  glutamate  or  1.5  M  Cl~ 
extracellularly  (90nA,  300  ms).  Mean  peak  responses  to  click  ^  3  sd  above  mean 
baseline  levels  of  activity  were  found  only  during  the  initial  ten  click 
presentations  and  after  the  ten  presentations  of  c  t  +  g.  Some  units 
responsive  to  click  could  be  found  within  each  group  of  cells  tested.  Among  the 
responsive  cells,  responses  to  the  ten  "test"  clicks  were  largest  after 
presentation  of  c  +  t  g,  with  or  without  subtraction  of  mutual  baseline 

activity.  All  but  one  of  the  cells  responding  after  c  ■*>  t  g  showed  increased 
activity  in  response  to  glutamate.  Our  results  show  that  application  of 
glutamate  after  click-CS  and  tap-US  can  produce  effects  on  cellular  activity 
resembling  those  found  after  adding  HS  to  the  same  CS  and  US.  This  evidence 
also  favors  the  hypothesis  that  glutamate  or  a  glutamate-like  chemical  is 
released  at  these  cortical  neurons  by  HS  and  that  the  resulting  increase  in 
activity  contributes  in  some  way  to  the  rapidity  of  conditioning. 


of  awake  cats.  Intracellular  iontophoretic  application  of 
calcium-calmodulin-dependent  protein  kinase  (CaFK)  was  followed  by  a  30  sec 
period  of  steady  depolarization  (1.0  nA).  These  cells  showed  an  increase  in 
input  resistance  in  comparison  with  a  control  group  of  fifteen  cells  given 
depolarization  only,  without  application  of  CaFK  (Fig.  1).  Post-iontophoretic 
measurements  of  input  resistance  in  cells  given  CaFK  alone  were  not  increased, 
nor  was  input  resistance  increased  in  cells  given  equivalent  negative  currents 
through  electrodes  containing  only  KCl.  A  detailed  comparison  of  results  in 
experimental  and  control  neuronal  populations  is  given  in  Fig.  1.  The  results 
indicate  that  intracellular  injection  of  calcium-calmdulin-dependent  protein 
kinase,  followed  by  depolarization  and  depolarization-elicited  impulse  activity, 
transiently  increases  input  resistance  of  neurons  of  the  motor  cortex  of  cats. 
Depolarization-induced  discharge  was  needed  to  change  the  membrane  response  of 
cortical  neurons  to  acetylcholine  or  cyclic  GMP  from  a  transient  to  a  persistent 
one  (cf..  Woody  et  al.,  1978).  An  analogous  increase  of  input  resistance  can  be 
produced  in  the  Type  B  photoreceptor  of  Hermissenda  by  applying  protein  kinase 
and  sufficient  depolarization  paired  with  light  to  increase  calcium  conductance 
and  internal  calcium  concentration.  It  appears  that  some  of  the  same  control 
Biechanisms  responsible  for  elaboration  of  associatively  induced  behavioral 
changes  in  Hermissenda  may  be  operative  in  neurons  of  the  cat  motor  cortex  that 
support  the  perforsiance  of  the  learned  motor  tasks  that  we  are  studying  (Woody, 
Alkon  and  Hay,  Brain  Res..  1984). 


I 
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Figure  1.  A)  Resu^^s  of  intracellular  injection  of  Ca  -calmodulin-dependent 
protein  kinase  (Ca  -CAMdPK)  followed  by  application  of  depolarizing  current* 
Input  resistance:  (i  and  iv)  »  before  protein  kinase  injection;  (ii  and  v) 
after  protein  kinase  injection;  (iii  and  vi)  ■  immediately  after  subsequent 
depolarization,  (i,  ii,  and  iii)  show  the  maintenance  of  bridge  balance  to  null 
out  changes  in  electrode  resistance.  In  A  (ii)  the  bridge  is  slightly  out  of 
balance.  An  increase  in  resistance  is  seen  in  iii  and  vi.  Calibrations  are  as 


I 

y 


indicated  below  and  to  the  right  of  this  portion  of  the  figure.  12  ms  applies 
to  both  time  calibrations  and  the  20  mV  calibration  applies  to  all  records. 

(The  tops  of  the  spikes  are  cut  off.)  B)  Bar  graphs  of  average  change  in  input 
resistance  before  and  after  protein  kinase  plus  depolarization  in  responsive 
cells  (PK  +  DP),  after  protein  kinase  only  (PK  only),  after  depolarization  alone 
(DP  only),  and  after  passage  of  negative  current,  i.e.,  hyperpolarization, 
without  iontophoresis  of  protein  kinase  (KCl  HP).  Standard  errors  of  the  means 
are  shown  for  each  different  group  of  cells. 
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udies  of  intrace llularlv  recorded  neurons 


rtex  of  conflcious  cats  were  made  in  coniunction  with  intracellular 


iniections  of  HEP  (e.a..  Fig.  2).  Stable  recordings  characterized  by  action 
potentials  of  amplitudes  smaller  than  the  recorded  resting  potentials  were 
correlated  with  recoveries  of  injected  dendrites.  Penetrations  with  dendritic 
recoveries  had  higher  input  resistances  than  did  those  with  recoveries  of  both 
somas  and  dendrites.  Increases  in  spike  height  during  pressure  injection  were 
greater  in  recordings  with  dendritic  recoveries  than  in  recordings  with 
recoveries  of  both  somata  and  dendritic  processes  (Woody  et  al.,  J. 
Neurophvsiol..  1984). 

Additional  studies  assessed  possible  injury  arising  from  cell  penetrations. 
The  response  of  penetrated  neurons  to  repeated  click  stimuli  was  compared  with 
that  of  unpenetrated  (extracellularly  recorded)  units  of  the  same  cortical 
region.  Responses  obtained  from  penetrated  neurons  were  separated  into  4  groups 
according  to  the  size  of  the  recorded  action  potential.  The  magnitude  of  the 
response  to  click  was  much  the  same  in  cells  with  action  potentials  ranging 
between  50  -  60  mV,  40  **  50mV,  and  30  40  mV.  The  magnitude  was  slightly 

greater  in  the  group  with  action  potentials  ranging  between  20  -  30  mV  .(suggest¬ 
ing  some  slight  depolarizing  injury  to  some  of  these  cells).  The  response 
profiles  were  comparable  to  those  of  extracellularly  recorded  units  (Woody  et 
al.,  2.*  Heurophvsiol. .  1970;  Woody  and  Engel,  J .  Neurophvs io  1 . .  1972).  Studies 
using  K-*-  ion  sensitive  microelectrodes  indicated  that  "intracellular"  recordings 
were  in  fact  made  intracellularly.  It  appears  that  whatever  injury  arose  from 
the  penetrations  of  these  cells  was  minimal  and  was  not  sufficient  to  impair  the 
ability  of  most  cells  to  respond  with  spike  activation  to  natural  stimuli  such 
as  weak  click  (Woody  et  al.,  J,  Neurophvsiol. .  1984). 

6.  Effects  of  local  increases  in  membrane  resistance  on  current  spread  in 


ndrites  were  explored  using  a  passive  cable  model  for 


determining  the  transient  potential  in  a  dendritic  tree  of  known  geometry.  The 
morphology  was  obtained  from  a  montage  composed  of  photomicrographs  taken  at 
different,  overlapping  areas  within  serial  sections  of  an  HRP-injected,  layer  V 
T"'Tamidal  cell  of  the  cat  motor  cortex.  A  passive  cable  model  which  could 
determine  the  transient  potential  in  dendritic  trees  of  arbitrary  geometry  was 
used  to  examine  the  efficacy  of  different  loci  of  increased  membrane  resistance 
for  given  loci  of  current  injection.  The  model  used  the  passive  cable  equation 
(cf..  Rail,  1962)  to  express  the  potential  for  each  interbranch  segment  of  the 
dendritic  tree.  By  matching  boundary  conditions  at  branch  points  and  termina¬ 
tions,  a  system  of  equations  was  readily  obtained  for  the  Laplace  transform  of 
the  potential  at  the  ends  of  each  segment.  The  inverse  transform  could  then  be 
quickly  computed  for  any  arbitrary  time  point.  Since  only  one  equation  was  re¬ 
quired  for  each  interbranch  segment,  this  approach  used  far  fewer  equations  than 
the  compartmental  approach.  Using  this  model  it  was  found  that  an  increase  in 
membrane  resistance  in  the  region  immediately  proximal  to  the  point  of  current 
input  was  more  effective  in  increasing  soma  potential  than  an  increase  in  a 
comparable  membrane  area  of  a  more  proximal  dendritic  region.  Under  certain 
circunstances  a  distal  increase  in  membrane  resistance  could  be  more  effective 
than  a  comparable  proximal  increase  depending  on  the  locus  of  current  injection 
and  the  morphology  of  the  dendritic  tree.  Tests  with  this  model  support  the 
view  that  increases  in  membrane  resistance  could  produce  the  increases  in  neural 
excitability  found  in  these  cells  after  conditioning  and  could  account  for  the 
increase  in  activity  of  these  neurons  in  response  to  the  auditory  CS  (Holmes  and 
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7 .  Current-voltage  relationshiofl  of  oericruciate  cortical  neurons  of  awak 


cats  were  studied  in  vivo.  Current  pulses  ranging  from  +,  4nA  and  of  30-500  ms 
duration  were  injected  in  17  pericruciate  neurons  in  order  to  investigate  their 
current-voltage  relationships.  An  active  bridge  circuit  was  used  to  inject  the 
current  pulses.  It  was  important  that  the  circuit  be  accurately  balanced  so 
that  the  measurements  would  not  be  distorted  by  changing  electrode  resistance. 
Therefore,  an  algorithm  based  on  the  method  of  Takeuchi  et  al.,  1981,  was  used 
to  balance  the  bridge  circuit  automatically.  It  required  only  that  the 
electrode  time  constant  be  much  shorter  than  the  cell  time  constant.  Current 
monitor  and  intracellular  voltage  records  were  A/D  converted  and  analyzed  (using 
a  threshold  detection  method)  to  determine  the  time  of  current  pulse  onset  and 
offset.  The  intracellular  voltage  record  was  sampled  just  before  pulse  onset 
and  just  after  charging  of  the  electrode  (i.e.,  about  50-100  usee  after  pulse 
onset),  the  difference  in  the  two  measuranents  being  the  magnitude  of  bridge 
imbalance.  This  difference  was  then  subtracted  from  the  voltage  trace  for  the 
duration  of  the  current  step,  and  the  data  D/A  converted  and  displayed  oscillo- 
graphically.  The  slope  of  the  IV  plot  in  the  hyperpolarizing  region  was  taken 
as  the  best  estimate  of  input  resistance.  It  averaged  9.1  megohms  across  the  17 
neurons.  The  cells  had  a  mean  resting  potential  of  62  mV,  and  a  mean  action 
potential  height  of  59.6  mV.  Rectification  was  not  detectable  in  the  range  of  * 
0.5  nA  current  injection  in  94Z  of  the  cells  (Berthier  and  Woody,  Soc.  Neurosci. 
Abstr. .  1983). 


8.  Effects  of  PT  stimulation  on  PSP  production  were  studied  in  intracell 


ular  recordings  from  62  cells  of  the  motor  cortex  of  awake  cats.  (Woody,  et  al. 
Brain  Res. .  1985).  Of  these  cells,  10  showed  an  IPSP  that  decreased  with 
hyperpolarization  and,  in  5  of  the  10  cells,  the  IPSP  was  reversed  with 
additional  hyperpolarizing  current.  In  9  of  the  cells,  it  was  possible  to 
sieasure  a  decrease  in  resistance  at  the  time  of  the  IPSP.  This  IPSP  has  been 
recognized  previously  by  other  investigators  and  is  thought  to  reflect  an 
increase  in  chloride  conductance.  In  30  of  the  remaining  cells,  a  quite 
different  IPSP  was  found  during  the  same  35-120  msec  period  following  PT 
stimulation.  In  each  of  these  cells,  the  IPSP  increased  in  size  with  the 
application  of  hyperpolarizing  current  and  could  not  be  reversed  with 
hyperpolarization.  With  depolarizing  current  the  IPSP  decreased  in  size.  The 
resistance  was  measured  at  the  time  of  the  IPSP  by  comparing  the  magnitude  of  a 
continuously  repeated  (20  ms  on,  20  msec  off)  bridge  pulse  during  the  IPSP  with 
that  prior  to  the  PT  shock  that  elicited  the  IPSP.  An  increased  resistance  was 
found  to  accompany  the  IPSP.  Conductance  decrease  IPSPs  were  seen  in  these 
cells  irrespective  of  whether  antidromic  spikes  were  produced  by  PT  stimulation. 
Conductance  decrease  IPSPs  have  been  reported  previously  (Siggins  et  al,  1971; 
Engberg  and  Harshall,  1971;  Smith  and  Weight,  1977),  but  not  in  neurons  of  the 
motor  cortex.  (PT  stimulation  is  an  effective  US  in  producing  conditioned 
behavior  [O'Brien  et  al.,  1977].) 


9«  Cybernetic  considerations  relevant  to  a  theoretical  approach  to 
analysis  of  neuronal  adaptation  in  a  nerve  network^  excerpted  from  a  book  on 
Memory.  Learning,  and  Higher  Function  by  Dr.  Woody,  are  enclosed  as  an  Appendix* 
Some  aspects  of  that  material  may  be  summarized  as  follows: 

An  adaptive  system  can  be  described,  cybemetically,  as  a  system  that 
modifies  its  'internal  structure  as  a  function  of  experience,  thereby  altering 
the  system  operation.  Ordinarily,  the  system  operation  will  become  increasingly 
optimized,  by  means  of  feedback,  in  the  approach  to  some  operational  goal.  In 
this  context  goal-seeking  will  be  the  process  by  which  the  component  or  adaptive 
element  moves  toward  or  maintains  a  particular  system  state*  A  key  feature  of 
any  adaptive  system  will  be  the  features  controlling  the  adaptation.  The 
control  sub-system  may  or  may  not  require  associated  memory.  If  so,  the  memory 
may  evolve  in  a  trivial  or  non-trivial  fashion,  with  or  without  variation  in  the 
original  set  point.  Control  of  goal-seeking  may  be  expected  to  be  accomplished 
by  means  of  feedback.  The  latter  will  ordinarily  involve  some  closed-loop 
operations.  Interestingly,  a  great  many  psychophysiological  formulations  of 
adaptive  neural  systems  have  neglected  to  specify  closed-loop  operations  by 
which  such  feedback  could  be  accomplished  as  opposed  to  open-loop  operations 
which  do  not  lend  themselves  to  modification  of  the  involved  element  as  a 
consequence  of  the  element's  past  adaptation  (cf.,  Kandel  and  Spencer,  1968). 

Physiologically,  many  adaptive  cellular  systems  lend  themselves  to 
closed-loop  goal-seeking  processes.  These  range  from  biochemical  feedback  loops 
(within  the  metabolic  context  of  the  cell  itself)  to  recurrent  collateral 
systems  with  relatively  direct  feedback  as  well  as  indirect  feedback  through 
more  extensive  polysynaptic  networks  (cf.,  Rasmussen  and  Goodman,  1977; 

Phillips,  1974).  At  the  level  of  cellular  components  in  the  brain,  there  exist 
several  candidate  mechanisms  for  the  control  of  neural  adaptation: 

i)  The  "Tin-Tang"  hypothesis  has  been  advanced  in  which  so-called 
excitatory  and  inhibitory  neurotransmitters  could  control  closed-loop 
goal-seeking  adaptations  depending  upon  neuronal  conductance  changes  by  means 
of  intracellular  second  messengers  such  as  cyclic  AMP  and  cyclic  GMP.  The 
cyclic  nucleotides  are  thought  to  interact  reciprocally  to  facilitate  either 
excitatory  or  inhibitory  effects  (Bloom,  1975,  1976;  Goldberg  et  al*,  1973). 

ii)  The  principle  of  voltage-dependent  control  of  neuronal  spike 
activity  is  well  established.  The  possibility  arises  of  voltage  dependent 
induction  or  potentiation  of  cyclic  nucleotide  release  as  well  as  the 
likelihood  of  coupled  sodium  or  potassium-calcium  channels  with 
voltage-dependent  features  (Loewenstein,  1975;  Lux  and  Eckert,  1974;  Heyer 
and  Lux,  197 6a, b). 

iii)  Entrainment,  i.e.,  the  production  of  multiple  spike  discharges 
encroaching  upon  relative  refractory  periods,  might  furnish  a  chemical  signal 
for  cellular  mechanisms  controlling  neural  adaptation,  particularly  after 
associative  stimulus  pairings  as  in  conditioning.  In  cortical  neurons, 
entrainment  is  probabilistically  an  uncommon  event  in  contrast  with  PSP  or 
spike  production,  per  se.  resulting  from  natural  auditory  stimuli  ^ich  serve 
as  CS's  in  Pavlovian  blink  conditioning  (Woody  et  al.,  1970;  Engel  and 
Woody,  1972).  Other  evidence  (Woody  et  al.,  1976)  indicates  that  entrainment 
might  interact  with  acetylcholine  or  cyclic  GMP  to  control  aspects  of 
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persistent  adaptation  in  mammalian  cortical  neurons. 

The  practical  significance  of  using  a  closed-loop  cybernetic  approach  to^ 
understand  cellular  adaptation,  even  at  the  biochemical  level,  is  just  beginning 
to  be  re-evaluated  and  appreciated.  For  example,  the  following  is  excerpted 
from  a  review  article  by  Rasmussen  and  Goodman  (1977). 
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(From  Physiological  Reviews) 


PAGE  14 


Systems  of  this  type  are  of  course  restricted  in  the  type  of  operations 
they  can  perform  and  the  geometric  patterns  that  can  be  recognized.  For 
example,  such  systems  cannot  compute  connectedness  of  geometric  figures,  whereas 
they  can  compute  convexity  and  related  processing  operations  of  the  type  called 
local  or  conjunctively  local  by  Minsky  and  Papert.  Humans  may  not  be  able  to 
compute  some  forms  of  connectedness  either: 

e.g. : 


The  possibility  exists  that  modification  of  Uttley's  algorithm  can  result 
in  the  introduction  of  a  self-class if vine  input.  By  self-classifying  input  is 
meant  an  input  of  particular  functional  significance  which  is  identifiable, 
within  the  adaptive  element,  by  means  of  its  stochastic  pattern  of  appearance 
alone.  Moreover,  this  stochastic  pattern  need  not  unduly  disrupt  the  overall 
function  of  the  adaptive  element's  operation. 

The  informon  model  of  an  adaptive  neural  element  (Uttley,  1976) 
incorporates  classifying  inputs,  closed-loop  feedback  concerning  the  operational 
state  of  the  element,  and  an  appreciation  of  goal-seeking  in  the  algorithm 
regulating  useful  adaptation.  Several  constraints  are  particularized  that  are 
critical  if  the  informon  is  to  successfully  discriminate  one  input  from  another. 
These  are  a)  the  algorithm  by  which  the  weightings  of  synaptic  inputs  are 
altered,  b)  the  need  to  achieve  system  normalization  through  negative  (not 
positive)  feedback  of  information  regarding  the  current  system  state,  and  c)  the 
need  for  a  classifying  input  to  distinguish  or  identify  which  input  signal  is 
the  particular  signal  to  be  discriminated.  Tests  of  this  model  have  found  that 
each  of  these  constraints  is  required  for  the  element  to  adapt  usefully. 

Synaptic  weighting  is  altered  according  to  the  Shannon  mutual  information 
function  between  certain  synaptic  inputs  in  combination  with  closed-loop 
negative  feedback  reflecting  the  element's  internal  state.  In  summary,  it  would 
appear  that  there  are  empirical  as  well  as  theoretical  reasons  why  "smart" 
adaptive  elements  need  to  incorporate  goal-seeking  as  well  as  closed-loop 
feedback  into  their  design. 

In  some  adaptive  networks,  input  analysis,  i.e.,  the  processing  of 
sensory- label led  information  (cf.,  Mountcastle,  1974),  is  explicable  in  terms  of 
the  group  invariance  theorem  of  Minsky  and  Papert.  This  theorem  permits 
analysis  of  operations,  such  as  the  geopmetry  of  certain  sensory  image 
processing,  by  algebraic  means  instead  of  statistics  and  thereby  reverses  a 
trend  in  this  field.  The  group  invariance  theorem  examines  the  relationship 


between  all  possible  receptor  activations  (all  sets  of  sensory  labels)  and  their 
representation  across  the  theoretical  space  of  an  adaptive  network,  given 
certain  architectural  constraints.  This  result  is  a  description  of  an  orderly 
relationship  in  which  no  matter  how  complexly  the  network  is  organized,  the 
space  required  for  a  particular  sensory  labelling  can  be  specified.  In  summary 
form,  the  group  invariance  theorem  states  that  if: 
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Scuatioa  3.6 


Research  supported  by  AFOSR  (1978-1982) 

1.  The  sampling  distribution  of  neurons  obtained  bv  our  intracellular. 
cortical  recording  procedure  was  investigated.  The  sample  of  HRP-identif ied 
neurons  was  found  to  be  essentially  equivalent  to  that  seen  in-situ  (determined 
from  Golgi-stained  sections  of  these  cortical  regions).  Seventy  percent  (70Z) 
of  penetrations  were  of  cells  in  layers  III  and  V,  and  70Z  of  the  penetrations 
were  of  pyramidal  shaped  cells.  There  was  a  slight  tendency  to  over-sample 
neurons  with  extensive  dendritic  arborizations.  Samplings  of  every  major 
morphologically  identified  in-situ  neuronal  type  were  obtained  by  our 
electrophysiological  procedures  (Sakai  et  al..  Brain  Res. .  1978). 

2.  The  response  properties  of  penetrated  neurons  to  injected  polarizing 
currents  were  investigated  and  found  to  be  normal.  The  accommodative  response 
to  ramp  depolarizing  currents  was  assessed;  most  responses  were  of  the  simple 
type  rather  than  ceiling  or  minimal  gradient,  (cf.,  Koike  et  al.,  Exn.  Br.  Res. . 
1968a, b).  Normal  I-V  plots  and  input  resistance  were  also  obtained.  Several 
lines  of  evidence  suggested  that  many  cortical  neurons  have  dendrites  that  do 
not  support  active  propagation  of  action  potentials  and,  instead,  serve  the 
integrative  process  of  neuronal  information  handling  (Woody  and  Gruen, 

Brain  Rea..  1978). 
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3»  In-vitro  calibrations  of  pressure  microiniection  techniques  were 
obtained.  Controlled  release  of  100  femtoliter  volumes  was  demonstrated.  A 
number  of  other  laboratories  are  adopting  this  technique  for  testing  local 
biologic  effects  of  pharmacologic  agents  (cf.,  Sakai  et  al.,  Neurooharmacol. . 


GMf  icGHFj  on  cortical  neurons  were  completed.  These  agents  appear  to  have 
similar  effects  on  input  resistance,  ACh  acting  extracellularly  on  cell  surface 
receptors  (of  muscarinic  type),  cGMP  acting  intracellularly.  The  input 
resistance  is  increased  transiently  by  the  effect  of  these  agents  alone  and 
persistently  by  application  with  cell  depolarization  sufficient  to  produce 
repeated  discharge  (Woody  et  al..  Brain  Res . .  1978). 

It  appears  that  neurotransmitters  act  in  a  dual  manner  in  these  cells,  as 
in  others,  to  convey  information.  One  action,  the  direct  "neurotransmitter 
effect",  serves  primarily  to  transmit  information  through  the  cell.  The  other 
action,  the  'Wdulatory  effect",  serves  to  control  adaptation  as  a  function  of 
the  information  transmitted.  The  two  actions  are  kept  separated  in  the. 
time-frequency  domain  by  different  time  courses  of  involved  biochemical 
pathways;  (cf.,  Rlopf,  A.H.,  Brain  Function  and  Adaptive  Systems  —  A 
Heterostatic  Theory.  AFCRL  Dept.,  H133,  1972). 

A  third  variable,  depolarization  included  discharge,  serves  to  make  the 
adaptation  persistent  rather  than  transient. 


1 1  ( •  J  I  ()  <•! 


With  low  rates  of 

current  spread,  graded  changes  in  threshold  produced  graded  changes  in  output 
discharge.  With  high  rates  of  current  spread,  the  neuron  became  a  bistable 
.(decisional)  operator  where  spiking  was  enhanced  if  the  threshold  was  below  a 
certain  level  and  suppressed  if  above  that  level.  The  enhancement  was 
considerably  more  pronounced  in  neurons  with  non^propagative  than  with 
propagative  dendrites.  With  propagative  dendrites  a  less  intense  input  was 
needed  to  initiate  somatic  spiking  (Levine  and  Woody,  Biol.  Cybernetics.  1978 


Similar  increases  in  input  resistance  were  obtained  with  this 
drug  as  with  ACh  and  the  effects  could  be  blocked  by  atropine  (a  muscarinic 
receptor  blocker).  One  of  the  cells  responding  to  aceclidine  with  an  increased 
resistance  was  identified  by  injection  of  BRP  as  a  pyramidal  cell  of  layer  VI 
(Swartz  et  al..  Proc.  West.  Pbarm.  Soc..  1978). 


pressure  injected  into  the  cells  after  studying  the  effects  of  ACh.  cGMP  was 
also  applied  intracellularly  by  pressure  injection.  Pyramidal  cells  of  layers  V 
and  VI  responded  to  these  agents  with  increases  in  resistance.  The  responsive 
neurons  included  those  of  layer  V  activated  antidromically  by  FT  stimulation. 
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A  comparison  of  the  results  of  pressure  injected  cGMF  with  those  of 
intracellularly  iontopboresed  cGMF  showed  similar  changes  in  resistance,  but  the 
increase  in  firug  rate  after  the  hyperpolarizing  iontophoresis  did  not  occur 
after  pressure  injection.  The  results  suggest  that  cGMP  and  acetylcholine 
produce  similar  effects  in  similar  neurons  of  the  motor  cortex,  the  primary 
effect  being  a  conductance  decrease.  The  increase  in  firing  rate  following 
application  of  acetylcholine  appears  to  be  a  separate  effect  of  this  agent, 
apart  fxom  that  supported  by  cGMP  as  a  second  messenger.  This  effect  may  arise 
from  excitation  of  surrounding  neurons  presynaptic  to  the  one  recorded  or  from 
other,  direct  conductance  effects  of  acetylcholine  binding  at  the  neuronal 
receptors.  (Swartz  and  Woody,  J.  Neurobiol..  1979;  Woody  et  al., 

Soc.  Neurosci.  Abstr..  1979). 

8.  Effects  of  low  frequency  PT  stiimilation  on  cortical  neural  excitability. 
Antidromic  stismlation  of  the  pyramidal  tract  has  been  used  successfully  as  a  US 
to  produce  conditioned  learning  (O'Brien  et  al.,  1977).  Effects  of  low 
frequency  4-6  Hz  PT  stimulation  (stereotax,  coord.:  F  3.5,  L  4.0,  H  4.5)  on 
cortical  neurons  were  investigated.  Cortical  cells  activated  antidromically 
responded  predominantly  with  reduced  excitability  to  intracellularly  applied 
current.  Cortical  cells  activated  transsynaptically  responded  with  increased 
intracellular  excitability.  Those  cells  failing  to  respond  showed  no  change  in 
excitability  during  the  5-15  minutes  tested.  (Tzebelikos  and  Woody, 

Brain  Res.  Bull..  1979). 


OS  presentations  on  rates  of  discharge  and  excitabilit 


(Brons,  Woody  and  Allon,  ^  Neurophvsiol. .  1982).  The  excitability  to  weak 
(nA)  extracellular  electrical  stimulation  was  measured  among  single  neurons  of 
the  pericruciate  cortex  of  awake  cats  as  a  function  of  behavioral  state.  Levels 
of  neuronal  excitability  were  compared  1)  after  classical  conditioning  of  a 
facial  movement,  2)  during  extinction  of  the  conditioned  response,  and  3)  during 
unpaired  presentations  of  conditioned  and  unconditioned  stimuli  (CS  and  US). 

Neurons  projective  to  facial  muscles  via  polysynaptic  corticofugal  pathways 
showed  decreased  levels  of  excitability  to  weak  extracellular  stimulation 
following  conditioning  with  forward  pairing  of  the  CS  and  US,  extinction  with 
backward  pairing  of  the  stimuli,  and  presentations  of  the  US  alone.  These 
changes  in  excitability  were  attributable  solely  to  the  effects  of  US 
presentation  and  were  not  distinguishably  different  during  either  conditioning 
or  extinction  of  the  behavioral  response.  Small  decreases  in  rates  of 
spontaneous  firing  were  found  to  accompany  the  decreases  in  neural  excitability. 

The  data  support  the  conclusions  that  significant  nonassociative  changes  in 
neural  excitablility  occur  during  conditioning  and  extinction  due  to 
presentations  of  the  unconditioned  stimulus.  These  changes  support  latent 
inhibition,  behaviorally,  and  the  mechanism  of  these  changes  is  different  from 
that  of  changes  in  postsynaptic  excitability  found,  after  conditioning,  by 
intracellular  stimulation  of  similiar  cortical  neurons  (Woody,  Fed.  Proc. . 

1982).  The  increased  excitability  to  intracellular  currents  facilitates 
performance  of  the  specific  type  of  motor  response  that  is  acquired  and  is  also 
latent,  awaiting  a  command  signal  that  will  cause  the  response  to  be  initiated. 
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Further  details  about  studies  of  rapid  learning. 

The  rapidity  of  acquisition  of  conditioned  motor  responses  was  determined 
after  adding  hypothalamic  stimulation  to  click  CS  and  glabella  tap  US.  Our 
analyses  showed  a  two  order-of^naenitude  acceleration  of  the  rate  of  acquisition 
of  a  blink  response  over  that  achieved  by  pairing  the  same  CS  and  US  without 
hypothalamic  stimulation  (Kim,  Woody  and  Berthier,  ^  Neurophvsiol. .  1983). 
Changes  in  the  patterns  of  activity  of  single  units  of  the  motor  cortex  were 
isomorphic  with  the  development  of  the  conditioned  response  (Woody,  Kim  and 
Berthier,  Seurophvsiol. .  1983). 

Additional  findings  were  obtained  following  completion  of  the  following 
computer  programs. 


Computer  Program 

The  program  consists  of  three  functional  units:  stimulus  presentation  and 
data  collection,  histogram  generation  and  display,  and  behavioral  analysis  and 
data  storage.  Conditioned  (CS),  unconditioned  (US),  hypothalamic  (HS),.  and 
discriminative  (DS)  stimuli  are  presented  in  a  timed  sequence  for  ten  second 
trials  of  adaptation,  conditioning,  extinction,  or  delayed  HS  paradigms.  Timing 
of  stimuli  can  be  generated  spontaneously  for  on  line  experiments  or 
synchronized  to  an  analog  tape  pulse  for  analysis  of  prerecorded  data.  During 
each  trial,  five  seconds  of  EMG  data  encompassing  all  stimuli  are  sampled  at  2 
ms  intervals  from  the  left  and  right  orbicularis  oculi  and  levator  oris.  Eight 
histograms  are  generated  from  the  data  and  displayed  four  each  on  Mime  100  and 
VT105  video  terminals.  The  histograms  are  averages  of  three  trials  and  are 
normalized  to  the  tallest  bin.  The  Mime  100  histograms  are  400  ms  displays 
encompassing  the  CS-US  period  for  each  EMG.  The  VT105  histograms  can  be 
dynamically  modified  by  keyboard  codes  which  can  center  histograms  around  any 
stimuli  for  any  Elffi  and  display  from  100  to  1600  ms  of  data. 

The  computer  detects  conditioned  E)H>  responses  using  the  criteria  that  3 
consecutive  samples  in  the  current  trial  plus  1  of  the  2  previous  trials  plus 
the  average  of  those  3  trials  must  exceed  5  standard  deviations  above  the  mean 
of  spontaneous  activity  sampled  for  400  ms  before  the  CS.  The  response  must  be 
detected  between  100  ms  after  the  CS  and  20  ms  before  the  US.  If  a  response 
based  on  these  criteria  is  found,  the  three  trials  are  individually  stored  on 
disc  while  no  response  results  in  three  trials  being  averaged  before  disc 
storage. 


Results 

The  results  of  training  cats  with  click  CS,  tap  US,  hypothalamic 
stimulation  (HS),  and  an  added  hiss  DS  are  shown  in  Figure  3.  They  indicate 
that,  with  this  paradigm,  discriminative  responses  to  the  CS  are  acquired  within 
9  trials.  The  rate  of  acquisition  is  two  orders  of  magnitude  faster  then  when 
HS  is  omitted  and  permits  intracellular  recording  from  cortical  neurons  while 
learning  takes  place.  The  latencies  of  the  CRs  range  between  100  and  300  ms. 

Additional  effects  on  conditioning  of  adding  hypothalamic  stimulation  (HS) 
to  classical  application  of  CS  (click)  and  US  (glabella  tap)  have  also  been 
examined: 


vti.  A..  i 
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a.  Learning  in  animals  given  stimulation  at  effective  hypothalamic  loci  was 
compared  with  failure  to  learn  in  animals  given  stimulation  at  ineffective 
hypothalamic  loci.  Figure  4  shows  some  effective  and  ineffective  loci 
(Kim,  Woody  and  Berthier»  Neurophvsiol. .  1983). 

b.  Patterns  of  cortical  unit  response  to  hypothalamic  stimulation  may  be 
predictive  of  an  effective  locus  of  hypothalamic  stimulation  for  producing 
enhanced  rate  of  learning.  Patterns  of  activity  such  as  those  shown  in 
Figure  5  have  been  seen  with  effective  hypothalamic  stimulation  (Woody, 
Kim,  and  Berthier,  Jj,  Neurophvsiol. .  1983). 


I>a0  LATENCY  RESPONSES 


Figure  3.  Rapidity  of  Conditionine  and  Latency  of  CR's 


Developoient  of  EMG  responses  of  different  latencies  to  CS  (Solid  line)  or  DS 
(Dashed  line)  in  8  cats  during  conditioning.  Responses  were  defined  as  EMG 
responses  of  greater  than  5  sd  above  the  pre~CS  (spontaneous)  mean.  During 
training,  CRs  increased  with  trials  reaching  asymptote  (74Z  CRs)  within  9 
trials.  Responses  were  classified  into  four  windows  (0-80  ms,  101-200  ms, 
201-260  ms,  261-300  ms;  top  to  bottom,  respectively).  Cats  made  more  responses 
to  the  CS  than  DS  when  responses  of  greater  than  101  ms  were  analyzed.  During 
extinction  cats  siade  more  responses  to  the  CS  than  to  the  DS,  but  by  the  ninth 
trial  of  extinction  there  was  little  responding  to  either  the  CS  or  DS. 


«  Most  effective 
•  Effective 
A  Ineffective 


Figure  4.  Loci  of  Brain  Stimulation 

Filled  stars  represent  hypothalsnic  sites,  stimulation  of  which  produced  rapid 
acquisition  of  discriminative  eye-blink  CRs  at  over  90Z  performance  level.  The 
sites  of  stimulation  that  resulted  in  discriminative  CRs  being  performed  less 
consistently  are  represented  by  filled  circles.  Bebaviorally  ineffective  sites 
of  stimulation  are  indicated  by  filled  triangles. 


Abbreviations:  Cd,  caudate  nucleus;  Ch,  optic  chiasm;  Cl,  internal  capsule; 
FX,  fornix;  GP,  globus  pallidus;  LH,  lateral  hypothalamus;  Th,  thalamus;  TO, 
optic  tract;  VMH,  ventromedial  hypothalamic  nucleus. 
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Figure  5.  Responses  to  hypothalamic  stimulation  (*  is  first  of  four  electrical 
pulses)  of  two  (A  and  B)  HRP~injected  pyramidal  cells  of  the  cat  motor—sensory 
cortex. 
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Woody 


Chapter  7 

Cybernetics:  A  Means  for  Analysis  of 
Neural  Networks 


Tht  d€fthpm€nt  of  tht  srarisricei  theory  of  commumcerion  is  a  landnuirk 
in  tho  kistofy  of  communicstion  dxtory.  Our  primary  concern  in  a  com* 
munication  or  conrroi  probiem  is  the  flow  of  mesuzes.  Since  the  centrei 
idea  in  die  sxanstied  theory  uthat  messsics  cndrtoise  shouid  be  considired 
esrer.dom  phenomena,  the  theory  incorporates  probability  theory  aid  gen- 
ereiized  harmonic  analyiis  in  its  four^tion, 

(Y..W.  Ue,  I960) 

Commonsense  approaches  to  an  understandins  of  "'higher  function**  are  use¬ 
ful  but,  as  we  have  seen,  are  basically  introspective.  Such  approaches  could  be 
ill  advised  if. used  analytically  because  of  their  intrinsic  susceptibility  to  errors, 
particularly  those  of  the  type  illustrated  in  Fig.  6.6.  Other,  more  objective 
means  must  be  found  to  analyze  the  complex  integrative  functions  of  neural 
networks.  As  our  knowledge  of  anatomically  and  physioiogicaily  based  mem¬ 
ory  and  learning  advances,  so  must  our  expression  of  this  knowledge.  The 
form  of  this  improved  expression  is  likely  to  be  mathematical  and  as  spedflc 
as  expression  of  our  modem  knowledge  of  genetics  and  the  genetic  codes. 
The  purpose  of  this  chapter  is  to  explore  some  of  the  forms  that  are  likely  to 
serve  for  expression  of  our  knowledge  of  memorv*  and  learning. 

Analysis  of  large  populations  of  neurons  can,  in  principle,  be  approached 
from  the  same  sundpoint  as  analysis  of  numerically  small  reflex  networks. 
The  analyas  requires  application  of  systems  approaches  from  engineering 
disciplines  plus  consideration  of  the  limiting  physiologic  constraints  that 
apply  to  each  system  analyzed.  In  addition,  since  large  populations  of  neuroiu 
deal  substantially  with  the  processing  of  information,  their  overall  analysis 
requires  concepu  from  information  theory. 

This  concluding  chapter  examines  possible  means  for  analyzing  complex 
systems  using  mathematics,  engineering,  and  physics.  The  approach  is  called 
systems  analysis,  but  when  applied  to  adaptive  systems,  it  is  more  properly 
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termed  cybernetics,  the  analytic  science  that  deals  with  the  control  of  infor* 
nution  processing  in  man  and  machine.  By  using  the  techniques  described 
herein,  a  rigorous  analysis  can  be  made  of  linear  information  processing 
systems  and,  pediaps,  of  some  nonlinear  systems  as  well  [582483,1086]. 
Some  reflexes  within  the  brain  can  be  assumed  to  behave  as  a  linear  system 
and  can  be  investigated  by  linear  systems  analysis  [429,1123].  Such  an 
analysis,  though  properly  applicable  only  to  theoretical  systems  meeting  strict 
statistical  criteria,  still  provides  the  most  useful  beginning  toward  a  rigorous 
analysis  of  complex  nerve  networks.  It  simplifies  the  complexities  of  the  net* 
warks,  leads  to  formulations  of  the  transforms  of  given  inputs  into  particular 
outputs,  and  generates  more  precise  traiuforms  than  those  presently  existing. 

A  few  suitable  models  of  information  transfer  in  the  brain  have  been  de> 
vrioped  that  are  amenable  to  mathematical  analysis  [cf.'18-22466,S38-841, 
883,1033'!  038].  The  most  elegant  of  these  deals  with  image  recognition,  i.e. 
the  transmission  and  processing  of  sensory  labeled  messages  arising  at  the 
receptors.  Minsky  and  Papert  [672]  have  transformed  certain  probletiu  of 
image  recognition  into  problems  of  geometrj'-a  transformation  that  elegantly 
sunplifles  many  problems  of  analysis.  Then,  they  have  devised  a  theorem,  the 
Group  Invariance  Theorem,  that  provides  a  general  analytic  solution  for  one. 
set  of  the  geometry.  In  application,  the  Group  Invariance  Theorem  (p.  388) 
adequately  describes  the  geometry  of  sensory  reception  for  the  components 
of  two  specific  models  of  elements  of  an  image  recognition  network,  the  per¬ 
ception  and  the  informon.  In  these  models,  as  in  thg  brain  itself,  line  labeling 
appears  to  be  the  key  to  following  the  flow  of  information  t.Hrougli  its  com¬ 
plex  tnnsformation  from  sensory  input  into  motor  output.  Flowgraphs  and 
linear  systems  analysis  are  also  helpful  in  this  regard. 

Each  of  these  analytic  approaches  properly  begins  by  considering  known 
constraints  on  the  system  to  be  analyzed.  Therefore,  before  discussing  the 
models  and  their  analysis,  some  constraints  on  information  processing  that 
any  useful  model  of  brain  function  should  satisfy  will  be  considered. 


Gsnstraints 

Time  Consuna  of  Neural  Information  Flow 

How  npidly  can  information  be  transnxtted  and  processed  within  the  CNS? 
Contiieiion  Tme  and  Tmnsmission  Dday 

As  noted  earlier  in  Chapter  2,  the  rate  of  nerve  conduction  is  a  function  of 
fiber  size,  with  large  axons  conducting  more  rapidly  than  small  axons.  Trans¬ 
mission  of  an  electrically  propagated  impuise  along  a  neuron  may  proceed  as 
rapidly  as  160  m/sec  in  the  dorsal  spiiracerebeilar  tract  of  the  cat  [369]  or  as 
slowly  as  0.5  m/sec  in  the  finest,  unmyelinated  a.xons  of  the  spinothalamic 
system  [693] .  Thus,  while  it  could  take  as  little  as  2  msec  for  propnoceptive 
information  concerning  hindleg  position  to  reach  the  cerebellum  of  the  cat 
(a  distance  of  about  320  mm),  it  could  take  as  long  as  640  msec  for  infor- 
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Automata  Design 


L  Roeaptors 


IL  Spatial  aignal'sarfal  procassor-tcmporal  coda 


in.  Spatial  tignal'parallai  procasaor-Spatial  coda 


A.B 


AB  )3a 
B  >U 
BO  ho 

* 

>  d  I 


ImmJ 


0 
BO 
B  . 


2B 

20 


3: 9, 


3: 


Oiscnmtnatas  B,  but  not  A  0 


-  2: 


Oiscriminataa  A,  B,  D  and  survives 
ablation  of  any  1  of  3  inputs  to 


1: 


Discriminates  A.  8.  0  and  survives 
ablation  of  any  1  of  3  inputs  to  a^ 
and/or  #2 


Fig*  7.5.  Examples  of  the  components  of  different  automata,  L  Receptor  ele^ 
ments  common  to  each  example.  The  uppermost  of  the  four  elements  is  re^ 
eeptiwe  to  (and  intersected  by)  the  letters  A  and  B  but  not  D,  The  lowest  ele^ 
mens  is  receptive  to  3  only.  The  receptivity  of  the  remaining  two  elements  is 
as  indicated.  !HV,  Automata  with  different  network  architectures:  //,  rericl 
time  dependent.  III.  parallel  perceptron,  IV,  parallel  pandemonium.  In  II  three 
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IV.  Fonturo  wolghtifig  and  analysia  plus  fssdback  control 


Swsioa  DM  tm^ywri  Mvgrelaf* 


V.  Fcsturo  dstoetlofi  and  evolution 


a^  *2*  various  analytic  operators: 

e.0.,  template  matentng.  summing, 
diftorentlating,  etc. 


Selection  is  based  on 
the  sum  Wf  of  the 
weightings  X/  of  the  ay 
Individual  features  analyzed 


different  coded  outpua  of  the  decisional  element,  d,  are  shown  to  the  right. 
Below,  the  ambiguity  of  this  coding  is  i7iia/rarad  when  its  beginning  in  time  is 
uncertain.  In  III  changing  d,  the  number  of  inputs  required  to  fire  the  second 
order  elements,  changes  she  discriminative  property  of  the  network  as  dis* 
cussed  in  the  text  and  summarised  in  the  diagrem  to  the  right.  In  IV  a  variation 
on  in  introduces  feature  detection  (e^g„  summation,  filtering,  etc,)  as  well  as 
feedback  control  (heavy  dashed  line)  into  the  circuit,  A  version  of  IV  is 
shown  below  (Y)  in  which  details  have  been  inserted  after  Selfridgt’s  pande^ 
monbim  [8831,  Selection  of  evaluation  of  the  individual  demons  for  permu* 
tation  is  based  on  W^,  the  sum  of  the  weightings,  of  the  individual  feature 
detectors,  Sj. 

The  (Sled  circles  represent  serial  time  delay  elements.  The  elements  com¬ 
prised  by  dashed  lines  represent  a  clocking  mechanism  that  keeps  tnck  of 
time  following  presentation  of  the  stimulus  pattern  at  the  receptors. 
Depending  on  which  receptor  is  activated,  a  time  coded  signal  1—,  -1-,  or 
—1  will  be  generated  at  the  decision-making  element,  d.  Note  that  if  track 
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is  not  kept  of  time  (see  Fig.  7.5,  II,  lower  right),  the  code  becomes  am* 
biguous.  Though  the  example  is  oversimplified,  it  is  quite  representative  of 
the  type  of  processing  that  is  widely  used  in  digital  computers. 

The  term  parallel  processing  was  originally  used  to  designate  performance 
of  the  same  processing  operation  by  more  titan  one  channel  at  %  time.  The 
purpose  was  to  support  majority  logic  and  redundant  signal  processing  as  dis¬ 
cussed  tzt]xtxMultiplexed,  parallel  processing  resembles  much  of  the  process¬ 
ing  done  by  the  brain.  In  the  example  shown  in  Fig.  7  III,  the  second-order 
elements  receive  redundant  messages  and  paraQel  processing  is  used  to  gener¬ 
ate  a  "^decisional*'  output  at  element  ""d".  As  can  be  seen,  this  type  of  pro¬ 
cessing  has  remarkable  sorting  or  discziminacive  properties  when  adaptation  is 
introduced. 

If  each  of  the  second  order  elements  in  Rg.  III  is  set  to  fire  when  three 
inputs  are  received  (3/3),  the  network  will  distinguish  B,  but  not  A  from 
D,  ie.,  a|  (fires)  «  B.  If  the  **threshoid**  is  reduced  to  discharge  upon  re¬ 
ception  of  two  inputs  (2/2),  the  network  will  then  discnminate  A,  B,  and 
D*  An  A  will  be  designated  by  no  discharge,  a  B  by  discharge  of  both 
elements,  and  a  D  by  discharge  of  the  lower  element,  Le.,  TiTn  •  A.  (or 
nothing  presented),  aia^  «  B,  a^a^  »  O.  Moreover,  the  network  will 
continue  to  function  despite  destruction  of  any  one  of  the  three  input 
lines  to  the  upper  element. 

Although  one  must  be>vare  of  making  exact  transpositions  between 
mechanical  aiul  physiologic  models,  many  of  the  same,  general  theoretical 
considerations  concerning  learning,  memory,  and  even  higher  function  apply 
to  machines  as  to  physiologic  systems.  The  machines  give  us  a  physical  model 
which  is  more  accessible  to  analysis  and  is  more  easily  studied.  Three  machine 
automata  stand  out  from  the  others  in  providing  insigheful  models  of  learning 
operations,  component  interactions,  and  the  constraints  thereof.  They  are  the 
perception,  pandemonium,  and  the  informon. 

PeTceptroQ 

The  perce^ron  represents  an  early  attempt  by  Rosenblatt  and  colleagues  to 
develop  a  learning  automaton  based  on  their  conceptions  of  brain  organiza¬ 
tion  [SSS'BAl].  In  this  device,  the  components  consist  simply  of  modiaabls 
elements  and  their  interconnections.  As  shown  in  Fig.  726,  a  is  the  sum  of 
the  components  each  weighted  by  When  the  weightings  are  modified 
(Aa),  the  system  can  adapt  to  distinguish  a  particular  input,  identified  when 
^  is  >  some  predetermined  value,  B. 

(7.2) 

where  Aa  reflects  adaptation. 

The  example  of  parallel  prooessing  (Fig.  7.S,  III)  can  be  viewed  as  a  per¬ 
ception  by  making  d^Z  and  considering  ai  and  as  ha^1ng  weighted  inputs 
depending  on  the  threshold  settings,  8^. 
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In  such  systems,  the  performance  of  pattern  recognition  can  be  adaptively 
improved  with  relatively  simple  algorithms  of  element  modification  as  illus- 
trated  in  Fig.  7.5  (additional  algorithms  are  listed  on  p.  39S).  Although  this 
can  be  done  among  elements  with  randomly  organized  conneaions,  os  the 
original  perception  demonstrated,  a  more  efficient  operation  will  be  provided 
by  a  noiuandomly  organized  network.  Thus,  the  organization  of  the  adaptive 
network  may  become  a  critical  vatiable  in  the  learning  operations  that  are 
performed  by  such  a  system. 


Pandemoiunm 

An  example  of  a  nonrandomly  organized  automaton  is  pandemonium  of 
Oliver  Selfridge  [883].  Its  organization  is  hierarchical,  being  characterized 
by  multiple  layeis  supporting  difTexent  operations  as  shown  in  Fig.  7.5,  IV 
and  V.  The  initial  layer  again  consists  of  simple  receptor  or  data  collecting 
elements,  termed  data  demons  by  Seifxidge.  The  second  layer  consists  of 
specialized  analyzers  or  computational  demons.  They  process  incoming  dau 
by  stereotyped  procedures  such  as  matched  fllteiing,.summat2on,  or  differen¬ 
tiation.  The  third  layer  consists  of  integrators  or  cognitive  demons.  They  inte¬ 
grate  weighted  inputs  from  various  compuutlonal  demons.  Finally,  a  decision 
maker  or  ""decision  demon**  selects  the  loudest  or  most  active  cognitiv-e 
demon(s)  and  by  its  (their)  identity  gives  priority  to  a  selected  set  of  receptors. 

Within  this  hierarchy,  adaptation  occurs  according  to  rules  of  reinforce¬ 
ment  specified  in  terms  of  the  effectiveness  of  each  element  in  performing  the 
selected  recognition  task.  Elements  which  are  more  contributory  to  successful 
image  recognition  are  positively  reinforced  by  increasuig  their  weighting.  Ele¬ 
ments  which  are  less  contributory  are  eliminated.  Permutations  of  the  analy¬ 
tic  algorithms  of  successful  elements  are  generated  to  replace  those  of  unsuc¬ 
cessful  elements.  Hill-climbing  techniques  are  used  to  secure  continued  im¬ 
provements  of  the  adaptations  with  extensive  attention  paid  to  the  problem 
of  avoiding  false  peaks. 

Several  insights  into  adaptive  information  processing  are  provided  by  pan¬ 
demonium.  Pandemonium  is  characterized  as  a  chaotic  operation  wtih  demons, 
subdemons,  and  sub-subdemons  shrieking  their  outputs,  adapting,  deciding, 
and  sometimes  evolving.  However,  the  diaos  turns  out  to  be  more  orderly  dian 
expected.  All  the  analytic  functions  arc  particulanzed  and  are.  to  a  significant 
degree,  predetermined.  Despiu  the  pm  depee  of  adapubUlty  vvlthin  the 
hieiarchy,  the  hierarchy  ts  relatively  Sxed.  The  reason  for  this  is  that,  ai- 
tbou^  the  adapubdity  pemuu  evolution,  it  is  along  a  predictable  pathway, 
and  occurs  withm  a  parucuJar  hiererdiy.  (This  feature  appears  to  hax-e  led 
this  particular  automata  to  a  paruculaiiy  lenaaous  pursuit  of  false  peaks 
during  hill-climbing  adaptive  operatiofia.)  Differences  m  the  design  of  the 
hierarchy  selected  for  Pandemoniu  i  ^renus  that  shown  in  Fig.  3.42  may 
tberefoie  be  of  some  consequence.  The  ability  to  switch  between  eiements 
may  need  to  be  matched  by  an  ab&bry  to  switch  between  hierarchies. 


PAGE  65 


346  Cybernetics:  A  Means  for  Analysis  of  Neurai  Networks 
Infonnon 

The  informon  model  of  Uttlcy  [1033-1038]  takes  a  somewhat  different 
approach  to  the  design  of  an  automaton,  concentrating  on  improving  the 
construction  of  the  fundamental  adaptive  element  itself.  The  basic  informon 
consists  of  a  single  element  with  multiple  inputs  F(XjJ  and  an  output  (Fig. 
7.6).  The  inputs  have  variable  weightings,  a.  One  of  the  inputs  is  defined  as  a 
reinforcing  input  Flz)  with  a  fixed  negative  weighting,  -k.  There  is  also  pro¬ 
vision  for  negative  feedback  of  information  concerning  the  operational  state 
of  the  element,  F(Y).  The  negative  feedback  is  required  for  stability  of  the 
adaptive  process.  F(Y)  is  some  function  of  the  output  of  the  element  prior  to 
the  state  of  binary,  spike  discharge.  There  is  finally  a  threshold  device,  6,  at 
or  just  before  the  output,  which  can  be  used  to  discriminate  between  different 
sets  ofinputs. 

Several  additional  variables  (or  constraints)  are  required  for  the  informon 
to  discriminate  successfully  one  particular  input  F{xi)  from  another, /’fx//). 
These  are: 

1 .  The  algorithm  by  which  a/  is  altered  (A  a). 

2.  The  need  for  a  reinforcing  input,  Ffr),  to  distinguish  or  identify  which  in¬ 
put  signal  is  the  particular  signal  to  be  discximinated. 

3.  The  need  to  a^eve  some  system  normalization  tlirough  negative  (not 
positive)  feedback  of  information  regarding  the  current  system  state, 
F(Y). 

Note  also  that  .by  picking  the  adaptive  algorithm  correctly  (e.g.,  Iqg  of  the 
mutual  information  between  inputs),  one  can  greatly  facilitate  both  normali¬ 
zation  and  input  discrimination. 

Algorithm  for 

The  trick  here  is  to  choose  an  algorithm  that  will  produce  S-shaped  adaptive 
operations  sudi  as  are  found  with  conditioning  or  other  simple  forms  of 
learning.  It  will  also  be  useful  to  have  a  decay  or  extinction  phase  of  zdify 
tation.  Adaptation  is  pei:fomied  by  changing  the  weighting,  a,  of  an  input. 

Simple  Informoo 


(Reinforcar) 


f(Yj 


^  Output 


Fig,  7.6.  Tht  basic  informon  element.  See  text  for  further  details.  (After 
Uttley/IOSSJ.J 
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Figure  7.7  shows  such  an  operation  graphically  and  enables  us  to  sec  how  a 
particular  choice  of  algorithm  may  or  may  not  produce  a  stable  change  in 
weighting. 

Uttlcy  points  out  that  analysis  in  the  phase  plane  between  change  in 
a(Le^  Aa)  and  a  itself  reveals  the  limitations  of  cenam  algorithms,  noubly 
those  proposed  by  Hcbb  [397]  and  by  Brindley  [97]  and  Marr  [646] .  This  is 
diowninFig.  7.8. 

Hebb*s  postulate  that  an  input  causes  an  increased  output  simply  indi¬ 
cates  that  if  a  is  positive  so  must  be  Acc  This  postulate  places  the  algo¬ 
rithm  for  acquisition  within  the  right  upper  quadrant  (•«-+)  of  Fig.  7.8,  but 
*  fails  to  specify  a  relationship  or  slope  between  variables  Aa  and  a  Brindley 
[97]  and  Marx  [646),  in  effect,  consider  a  pathway  with  two  states,  one 
initial  and  one  final,  in  which  Ao  and  o  increase  together.  With  limiting 
values  this  reduces  to  an  all-or-nonc,  two  state  process.  Wirhour  limiting 
values  this  represents  an  unstable  system  with  positive  feedback  which  will 
lead  to  regenerative  explosion  (line  in  Fig.  7.8).*  Uttley  picks  an 
algorithm  which  allows  the  values  of  Ao  and  a  to  fluctuate  in  the  manner 
shown  by  lines  “b”  and  ‘V  of  Fig.  7.8  ( I036j . 

System  Nomalization  by  FeedEccFbf  System  Stare 

-Uttley  points  out  that  regenerative  explosion  maybe  avoided  by  introducing 
a  normalization  process,  such  as  that  of  Malsburg  [1047],  However,  Mais- 
burg’s  type  of  normalization  shows  an  overly  restrictive  range  of  successful 

Adaptation  in  an  Informon 

a  Stable  a  Unstable 


Fy.  7. 7.  Adaptation  in  an  informon  involves  changes  in  a,  the  weightings  of  i 
input,  over  time.  In  the  example  to  the  left  an  increase  in  a  occurs  during  ac* 
quisition  of  input  facilitation  and  a  decrease  occurs  during  its  extinction  or 
de/it ciiittttion.  The  parallel  between  this  and  conditioned  behavior  is  deliberate. 

In  the  example  to  the  right  acquisition  is  an  unstable  process  with  a  declining 
unintentionally  past  a  certain  transition  point.  This  may  occur  because  of  fail* 
urt  to  regulate  the  system  state  appropriately  during  the  adaptive  process.  See 
text  for  further  details  about  regulating  the  system  state  (After  Uttley  { IQ36I.) 


*T7its  problem  is  avoided  in  some  nonlinear  systems  that  change  sute  as  levels  reach  cer* 
tain  limh& 
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Phase  Plane  Between  Aaand  a 


Fit.  7.8.  Phast  plank  of  a  versus  change  in  a  (ie..  6ct).  State  changes  along 
•*a'*  such  as  those  proposed  by  Marr  [646]  and  Brindley  [97)  are  unstable^ 
white  those  along  ••b"or  “c"are  not.  b  mirrors  the  state  change  during  acquir 
sition  in  Fig.  7,7;  ’•c'*  mirrors  the  state  change  during  extinciioTL  (After  Uttley 
[1036].} 

operation  when  applied  to  a  system  with  positive  feedback.  To  avoid  this, 
Uttley  turns  to  negative  feedback  as  shown  in  Eq.  7.3b.  Thus,  the  adaptation 
of  his  element,  and  probably  some  neuronal  elements  as  well,  depends  critical¬ 
ly  on  negative  feedback  of  information  concerning  the  sy^stem  state.  F(Y). 
Normaliiation  results  in  pan  from  the  negative  feedback  of  Information  con¬ 
cerning  the  system  state  (Fig.  7.9)  and  in  part  from  the  choice  of  adaptive 
algorithms  described  below  (Eqs.  7  Ja,  7  Jb,  and  7.5). 

tai^-kF[xi)F(Y).  (7Ja) 

where  F(Y}  «  Z  F[xi)(xi  and  ^  is  a  positive  constant. 

However,  this  is  still  not  enough  to  permit  successful  input  discrimination, 
which  depends  additionally  upon  introduction  of  a  reinforcing  input. 

Reinforeing  Input 

Reinforcement,  or  identification  of  the  particular  input  F(xi}  to  be  discrimi¬ 
nated  or  enhanced  by  increasing  a/,  is  done  by  introducing  a  separate, label¬ 
ing  input  F(z)  with  fixed  and  negative  (Eq.  7  Jb). 

Aa/  -^•kF(Xi)  «f  (7-3b) 

Given  an  input  F(x(),  otf  will  increase  if  F(z)  is  present  and  will  decease 
if  F(2}  is  absent.  With  repeated  reinforcement,  a,*  assumes  the  function  of  the 
acquisition  curve  shown  in  Fig.  7.7  (left)  with  Ao/  =  ct^ax  “ 
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Signlflcaace  of  Locus  of  Negative  Feedback  of  Information  Concerning  System  Stale 
Rdacivc  to  Level  at  which  the  System  State  Beonncs  a  Binary,  Allnsr-None  Output 


Syatam  A 


nogativo  feedback 


Region  of  binery 
I  transformation 


Adaptive 

element 

System  B 


negative  feedback 


Region  of  binary 
transformation 


Adaptive!® 
element  I 


••Not  OK’* 


Fif.  7,9,  By  chanzinz  the  locus  of  ne^riye  feedback  so  that  instead  of  scmp:, 
ling  the  internal  state  of  the  adaptive  element,  as  in  (AJ,  one  samples  only  the 
binary  output  of  the  adaptive  element,  as  in  fS),  one  loses  information  re¬ 
quired  for  normalization  and  an  unsatisfactory  adaptive  process  may  ensue. 
The  location  of  the  binary  encoder  is  shown  by  |1|.  (Cff  Uttley  [I033JJ 


Hg.  7.8).  Without  reinforcement,  Of  assumes  the  function  of  the  extinction 
curve  in  Fig.  7.7  fleft),  with  Aof  *  (line  V’  of  Fig.  7.8).  Without  a  rein- 
forcer,  F(zJ,  a  curve  such  as  that  shown  in  Fig.  7.7  (right)  would  be  obtained. 

The  transfer  properties  of  Uttley*s  adaptive  element  are  designed  then  to 
simulate  the  S-shaped  acquisition  curve  of  conditioning  plus  its  decrement 
during  extinction.  Considerable  attention  is  also  paid  to  controlling  and  limit¬ 
ing  elemental  adaptation  by  dosed  loop,  negative  feedback  of  the  element*s 
internal  state.  This  variable  provides  a  signiiicant  constraint  on  the  operation 
of  the  adaptive  element  and  may  constitute  a  general  requirement  of  success¬ 
ful  self-organizing  adaptive  operations. 

Mutual  Information  Constraint 

Uttley  imposes  one  further  constraint  on  the  operation  of  an  informon,  name¬ 
ly,  that  a  be  a  modification  of  Shannon's  mutual  information  function*: 


?(x^andy) 


■Kxi-.Y) 


I'l.ri 
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This  constraint  can  be  applied  to  the  operation  specified  in  Eq.  7  Jb.  As  a 
result: 

I 

+  pA)  \ 

or  to  simplify  . 

(7 J)  . 

Thus,  an  increase  in  Ffx^J  will  result  in  an  increase  in  ct/;  an  increase  in 
FfY)  will  also  increase  a/,  but  an  increase  in  F(Xf)  F(Y)  will  decrease 

In  summaiy,  parallel  processing  systems  with  adaptive  elemcTiU  appear  to 
handle  discrimination  tasks  quire  easily.  Hierarchically  organized  networks, 
such  as  pandemonium,  with  non-uniform  elements  and  specialized  adaptive 
properties  can  handle  some  forms  of  learning  with  particular  ease,  but  may 
ding  tenaciously  to  errors  in  discrimiitation  arising  from  their  particular  de¬ 
sign.  (This  erroneous  ‘^behavior*’  is  not  unlike  that  of  perseverauon  and  neg* 
lect  described  in  Chapter  6.)  Other  automata,  such  as  the  informon,  may  rely  • 
on  optimized  properties  of  more  uniform  adaptive  elements.  As  UttJey  has 
shown  (1036-1038},  the  adaptive  weightings  must  change  in  ways  that  are 
nonexplosive.  Introduction  of  negative  feedback  of  information  concerning 
the  state  of  the  controlled  system  can  contribute  to  a  normalization  process 
which,  in  turn,  can  reduce  the  p)cssibility  of  e.xplosive  change.  Ouhei  features 
such  as  relaxation  of  increased  weighting  and  discriminative  control  of  the 
weighting  changes  of  certain  inputs  require  additional  features.  These  may 
include  particularized  dependencies  between  inputs  such  as  the  mutual  infor¬ 
mation  feature  of  Uttley's  model  or  labeled  reinforcing  inputs  such  as  Ffz)  of 
Uttlcy*s  model.* 

By  slightly  redefining  Uttley’s  circuits  (Fig.  7.10),  it  is  possible  to  form 
dosed  loop,  positive  feedback  pathways  that  might  support  motor  labeling  in 
dassical,  assodative  conditioning  (see  Chapter  3).  Positive  feedback  would 
augment  a  particular  message  of  motor  agnificance  transmitted  within  a  spe¬ 
cific,  dosed  loop  circuit.  The  augmented  message  would  faciliiate  the  forma¬ 
tion  of  adaptations  along  the  pathway.  Another  mechanism  (e.g.  inactivation) 
would  be  required  to  avoid  explosive  change. 

Further  support  for  a  possible  role  of  positive  feedback  in  neural  control 
systems  is  furnished  by  Freeman’s  model  of  olfactory  bulb  circuitry  [299]. 

In  that  model,  the  effect  of  the  stimulus  is  to  increase  feedback  gain  in  an  en¬ 
semble  of  neurons  that  are  receptive  to  the  stimulus.  **lf  a  local  ensemble  con- 
Uining  sensitized  subsets  that  are  mutually  excitatory  is  excited,  the  basis 
exists  for  a  regenerative  increase  in  activity  in  response  to  an  adequate  stimu¬ 
lus’*  [299] .  The  model  has  five  main  features: 

*a  timpliflcatioA:  properly,  the  equation  incorporates  the  ensemble  avenge  of  the  fro- 
ouencies  of  sitniJ  occurrence.  See  Ottley  ( 1036] . 

f  Further  material  concerning  these  equations  can  be  found  on  pp.  381.382. 

*For  further  particularizations  of  interest,  see  L'ttlcy,  A.  M.  In  format  ion  7>ensmusion  in 
the Ntrifout  Sytttm.  London:  Academic  Press,  1979. 
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Analysis 


Feedback  of  Motor  Labeled  Information 


Motor  labeitr 


Fig,  7u0,  Schema  for  motor  (M)  Utbeied  reinforcements  derived  from  Uttley  s 
informon  model  The  US  activates  neurons  which  act  directly  (or  indirectly) 
as  the  Fit)  input-  Selective  labeling  of  the  ^upstream  *'  neurons  which  projea, 
selectively,  to  the  activated  units  is  potentiated  because  of  positive  feedback 
within  the  circuitry.  For  this  schema  to  operate  successfully,  some  feature 
sudi  as  local  recurrent  inhibition  would  be  required  to  monitor  the  system 
state  and  prevent  explosive  buildup  from  the  positive  feedback,  IS^eedback 
of  information  concerning  internal  system  state, 

lo  A  nonlinear  signal  range  that  is  near  linear  about  the  origin. 

2.  Bilateral  saturation  with  gain  approaching  zero  at  both  extremes  of  w*ave 
amplitude  (this  feature  provides  stability). 

3.  A  2:1  asymmetry  of  the  asymptotes  of  the  circuit  transfer  function 
(arising  from  the  features  of  the  olfactory  bulb  and  cortical  electrophysi* 
ology  on  which  the  system  is  modeled). 

4.  A  gain  that  increases  with  positive  (exdtatoiy)  input. 

5.  A  gain  that  is  modinahle  in  a  pattern  that  depends  on  background  or  steady 
state  activity,  which  in  turn  is  presumed  to  be  under  centrifugal  control. 

The  positive  feedback  should  satisfy  three  constraints  for  stability:  (1)  the 
regenerative  effect  should  not  be  unduly  perturbed  by  noise,  (2)  it  shoufd  be 
self-Itmicing  in  maximal  amplitude,  and  (3)  it  should  be  rapidly  selAterminating 
to  permit  additional  inputs  to  pass  [299]. 


Analysis 

Analysis  of  the  organization  of  systems  as  complex  as  the  brain  need  not  be 
considered  impossible  when  systems  involving  complex  communication  (tele¬ 
vision),  learning  (computer  automata),  elaborate  control  mechanisms  (guided 
missiles),  and  even  uncertainty  (the  atom)  have  proved  amenable  to  analysis. 
It  is  possible,  in  principle,  to  analyze  a  complex  system  if  it  is  fiiiite,  obeys 
the  laws  of  physics,  and  meets  the  constraints  of  the  analytic  method.*  This 


*Ofic  should  never  underestimate  the  importance  of  this  latter  consideration  (see  pp. 
322-401  and  Epilogue). 
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is  so  irrespective  of  whether  the  system  is  biological  or  mechanical.  Means  ex* 
ist,  such  as  linear  systems  analysis  (p.  366),  for  partitioning  many  complex 
systems  into  relevant  suboperations  that  are  easier  to  analyze,  and  some 
neural  ^sterns  are  amenable  to  this  focxn  of  partitioning  [429,1103,1123]. 
Other  means  such  as  flow  graph  techniques  (p.  36S)  can  be  used  to  analyze 
neural  network  operations  on  a  cell  to  cell  basis  despite  complex  interrela¬ 
tionships  including  feedback  between  receptor  and  effector  functions.  FinaUy, 
ineans  can  be  found,  as  by  computer  smulations,  to  reassemble  and  test  the 
analyzed  component  functions  with  reference  to  the  overall  organization  of 
the  network. 

Apart  from  complexity,  another  objection  that  is  frequently  raised  to 
analyzing  brain  function  is  that  general  ^ysical  theories  comparable  to  those 
found  in  chemistry  or  other  basic  disc^lines  are  lacking.  While  it  is  true  that 
theories  of  information  handling  are  not  so  advanced  as  those  in  other  fieids, 
the  existing  theories  have  been  found  applicable  to  predictive  treatment  of 
mfonnation  handling  by  real  systems.  Tlie  usefulness  of  Shannon's  infor¬ 
mation  coding  theories  in  the  commuiucations  industry  is  well  established 
and  has  been  complemented  by  the  emergence  of  additional  theories  in  the 
areas  of  systems  control.  The  chalienfe  for  neuroscientists  is  to  develop 
extensions  of  the  above  theortee  that  are  appitczble  to  treatment  of  specific 
neural  information  processing  systems.  The  basic  purpose  of  the  material  that 
follows  is  less  ambitious,  being  simply  to  outline  some  of  the  pctentialiy  rele¬ 
vant  analytic  methodciogies. 


Signal  Analysis 

The  fundamental  idea  of  Wiener  and  Lee*s  approach  to  analysis  of  communi¬ 
cations  systems  is  that  messages,  signals^  and  noise  should  be  considered  sta¬ 
tistically  and  described  in  terms  of  probability  theory  [532] .  Messages  are 
information  carrying  functions,  i.e.  member  functions  in  an  ensemble,  or 
numerically  large  aggregate,  of  signals  (relevant  information)  noise  (irrele¬ 
vant  information)  and  their*combinacion.  Communication  theory  has  led  to 
analysis  of  linear  message-transmission  systems  using  convolution  as  the  basic 
analytic  device.  Given  a  linear  system  (p.  396)  and  coruideration  of  signals  and 
noise  as  random  processes  [582],  signal  analysis  can  be  perfortned  by  time 
series  analysis  utilizing  (1)  Fourier  series,  (2)  power  spectral  density,  (3)  cor¬ 
relation,  and  (4)  convolution  (Table  72). 

Most  sigri^  to  be  analyzed  within  the  CNS  are  changes  in  voltage  or  cur¬ 
rent  as  a  function  of  time.  To  determine  die  structure  of  a  signal,  it  is  analyzed 
in  terms  of  its  frequency  components  (c.f.  Figs.  5.30,  7.11,  7.12,  and  7.14). 
The  signal  may  be  described  in  terms  of  its  major  frequency  components 
(Figs.  7.1 1,  7.12)  or,  more  precisely,  in  terms  of  the  power  consumed  across  a 
1  ohm  resistor  by  passage  of  the  difTerent  frequency  components  of  the  signal 
(including  harmonics).  The  latter  is  called  the  power  spectral  density  (Fig.  7. 14). 
Some  information,  that  concerning  the  phase  of  one  frequency  component 
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Control  of  Adaptive  Systems 

Linear  systems  analysis  admits  a  general  theory  of  adaptive  control  provided 
that  the  system  is  linear  and  the  usual  consuaints  are  satisfied.  One  constraint 
is'  that  the  result  of  adaptation  depend  on  the  entire  past  history  of  adapta¬ 
tion  within  the  system.  Another  is  (usually)  that  the  uansfer  function  of  the 
system  be  time  invariant. 

When  uang  this  theory,  adaptation  is  introduced  as  a  controller  function, 
gfrj,  as  in  Fig.  7.24.  It  operates  by  adding  an  additional  input  to  the  system 
much  like  F(Y),  the  negative  feedback  of  the  system  state,  in  Uttley's  infer- 
mon.  It  does  not  directly  modify  the  original  system  transfer  function.  Hfr). 
To  do  the  latter  would  lead  to  a  time-variant  or  self-organizing  adaptive 
system  which  could  easily  be  nonlinear  and,  therefore,  not  amenable  to 
ttialysis  by  this  theory. 

For  a  linear  system  with  the  feedback  circuit  shown  in  Rg.  7  24(.\),  the 
output,  Y(t)  is  a  function  of  the  input,  .^fr).  the  system  transfer  function,  H(tj 
and  the  controller  function,  gfr).  If  the  LaPlace  transfonn.  F(s),  of  each  func¬ 
tion  is  taken. 


e.g„  Output  F(y)  ■ 


Y(t)e‘^fdt,  (7.49) 


(note  relationship  to  Fourier  transform,  Eq.  7.6),  then,  for  a  linear  system. 


where 


/>*/j  ^FJs)F^(s). 


(7.50) 


Le„  in  the  absence  of  a  control  loop. 


F(y)*F{x)F(h). 


(7.51) 


The  output  of  the  linear  system  with  negative  feedback  (Fig.  may 

therefore  be  expressed  as 


F(y)» 


Ffh)Ffx} 

l+F(hJFfgj 


(7i2) 


For  the  feedforward  circuit  shown  in  Rg.  724(B), 


Ffy}^  l*F(gJ  F(x)F(h) 


(7.53) 


assuming  pos/nVe  feedforward. 

One  may  wish  to  consider  the  linear  control  circuits  of  Fig.  7.24.  the 
designs  of  automata  shown  in  Fig.  7.S,  and  the  algorithms  of  adaptation  listed 
on  p.  39S  in  relation  to  the  descriptions  of  control  sv'stems  that  follow. 
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Control  Systems 

Several  types  of  control  systems  are  recognized,  each  with  its  own  critical 
feature(s).  For  example,  there  are: 

Control  systems: 

1 .  Wixh  or  without  memory, 

2.  With  or  without  set  point  variance. 

3.  With  or  without  self-organizing  adaptation. 

4.  With  open  or  closed  loop  control 

5.  With  feedforward  or  feedback  control 

The  list  is  by  no  means  complete  or  (in  considering  how  to  classify*  different 
types  of  switches,  Oywheel  governors,  thermostats,  and  iiuiate  or  learned  be¬ 
haviors)  are  all  of  the  differences  unique  or  mutually  exclusive. 

Adaptations  involved  in  control  may  reach  some  maximum  or  minimum 
value,  or  may  proceed  at  some  steady  state  level  with  or  without  range 
bounding  as  was  described  earlier  (Figs.  7.7, 7.8). 

Open  Loop  Adaptive  Control  Systems 

An  open  loop  control  system  receives  no  feedback  information  regarding  the 
state  of  the  adaptive  system.  There  may  be  indirect  feedback  of  information 
(e*g.,  from  the  environment  and  changes  therein  caused  by  the  system*s 
operation)  to  support  the  predetermined  system  operation,  but  not  to  cause 
the  controller  to  adapt.  Conuol  is  exercised  entirely  by  predetermined 
adaptations  based  on  the  detection  of  prede lined  contingencies.  Tlius,  in  a 
thermosut  adaptation  occun  on.  the  basis  pf  temperature  detection  plus  a 
prespedfied  contingency  (if  the  temperature  is  low,  turn  on  the  heat;  if  high, 
turn  it  ofO*  There  is  no  feedback  to  alter  the  rules  of  adaptation  based  on 
past  performance.  There  is  instead  an  input  of  ambient  temperature  and  a 
fixed  course  of  adaptation  contingent  on  its  leNTl.  Neuronally,  open  loop 
adapution  may  be  contingent  on  two  different  synaptic  inputs  occurring 
together,  as  with  heterosynaptic  facilitation  and  inhibition. 

Open  loop  control  systems  will  typically  have  great  stability  since  their 
adaptive  features  are  entirely  predetermined.  However,  it  may  be  difficult  to 
achieve  a  control  operation  of  high  sensitivity  with  an  open  loop  s>*stem.  This 
is  because  the  accuracy  of  control  depends  on  the  system's  initial  calibration 
and  on  the  precision  of  the  involved  components.  The  operation  of  open  loop 
control  systems  will  be  vulnerable  to  component  breakdown  or  interference 
from  outside  noise  that  was  unantidpated  in  their  original  control  design. 
Driftage  away  from  the  initial  component  set  point  is  uncorrectable  with  an 
open  loop  control  system.  There  is  also  no  possibility  for  self-organizing  adap¬ 
tation,  since  there  is  no  regard  for  the  present  or  past  system  state. 

dosed  Loop^  Feedback  Control  Systems 

A  dosed  loop  control  system  normally  uses  feedback  concerning  the  value  of 
a  controlled  variable  or  the  state  of  the  adaptive  control  system,  as  a  means  to 
control  further  adaptation.  In  a  dosed  loop,  self-organizing  control  system, 
the  response  of  the  modified  element  should  have  a  direct  effect  on  the 
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control  action  (Fig.  7^4A).  This  circuitmay  be  compared  with  that  of  a  feed¬ 
forward  control  system  (Fig.  724B)  in  which  information  from  the  input 
modifies  the  controller  without  regard  to  the  system  state. 

Either  feedback  or  feedforward  drcuitiy  can  be  used  to  reduce  the  error 
or  improve  the  response  time  of  linear  oontrol  operations,  such  *as  deschbed 
eiulier,  and  the  drcuitries  may  be  either  positive  or  negative.  Because  of  the 
dosed  loop  operation,  feedback  may  have  self-po ten tia ting  effects  when  it 
operates  either  as  a  supplemental  control  input  to  the  system  or  as  a  seif- 
oiganizing  modifier  of  the  system’s  original  transfer  function.  Positive  (re¬ 
generative)  feedback  is  distinguished  from  negative  (degenerative)  feedback  in 
that  the  former  augments  the  gain  of  the  loop  system  and  can  lead  to  explo¬ 
sive  buildup.  Positive  feedback  returns  an  output  to  the  input  so  as  to  add 
another,  positive  input  This  will  permit  rapid  change  or  increased  sensitivity 
of  the  ^stem  by  which  transforms  between  input  and  output  are  performed; 
however,  it  also  tends  to  unstabilize  the  system  and  increase  distortion  of  the 
signal  input  Aegur/ve  feedback  returns  the  output  to  the  input  in  such  a  way 
as  to  add  another;  negative  input.  Negative  feedback  then  decreases  the  gain 
of  the  loop  system  and  can  lead  to  damping  or  a  cut  off  of  signal  transmission. 
This  tends,  to  stabilize  the  transfer  between  input  and  output  and  reduce  distor¬ 
tion,  although  the  sensitivity  and  rapidity  of  the  transfer  operation  may  be 
reduced. 

Negative  feedback  control  systems  have  a  system  response  that  is  reiamely 
insensitive  to  brief  external  disturbances  and  to  internal  vahations  in  para¬ 
meters  of  the  operations  controlled.  This  is  because  the  output.  e.z.,/7yy  in 
Eq.  7.52,  approaches  F(x)  ^  if  Ffh)  Fig)  >  1.  Thus,  small  deviations  in 
component  operations  or  even  the  originai  control  parameters  may  no:  overly 
disturb  the  control  system,  provided  that  their  manifestatior4S  are  accessible 
to  the  control  loop.  This  permits  relatively  noisy  components  to  be  used  for 
the  system  operation.  Note,  however,  that  when  a  dosed  loop  system  is 


(A)  Feedback 
MfO 


(B)  Feed  forward 
H(t) 


K  X(tf 

Y(0  ^  XfO 

i 

! _ _ 

Fig.  7,24.  Linear  systems  with  feedback  control  (A)  and  feedforward  control 
(Bf  The  systems  have  input  X(t),  transfer  function  H(t)  and  output  Y ft), 
gft)  and  lit)  are  the  controller  functions.  Differences  between  applying  the 
output  of  the  controller  function  as  an  **extra  ( additive )  system  input  versus 
applying  it  to  direct  adaptation  of  che  system  transfer  function  are  discussed 
in  the  text. 
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carrying  a  range  of  frequencies. over  the  feedback  path,  the  frequency  charac* 
teristics  of  the  network  may  become  an  important  source  of  error.  At  one  fre¬ 
quency  the  phase  of  the  signal  fed  back  may  be  such  as  to  produce  negative 
feedback,  but  at  another  frequency  the  phase  relationships  may  be  such  as  to 
cause  positive  feedback,  and  oscillations  may  occur.  Stability  of  control  can 
therefore  be  a  problem  with  closed  loop  control  operations,  since  with  closed 
loop  adaptive  control,  there  may  be  oscillating  errors  of  ovsrcorrection  leading 
to  explosive  instability  or  drift  in  an  undesired  direction.  Tlte  latter  feature, 
taken  in  a  converse  maimer,  lends  itself  to  self'Organizing  adaptive  control, 
provided  that  some  means  be  found  to  avoid  maladaptation. 

Some  typical  chancteiistics  of  closed  loop  systems  which  .may  be  of  inter¬ 
est  with  regard  to  their  possible  use  in  the  design  of  seif-organizing  systems 
aie  as  follows: 

1.  Some  stable  closed  loop  systems  tend  to  have  a  transient  response  per¬ 
formance  which  can  be  predicted  from  the  steady-state,  closed  loop  plot 
of  magnitude  versus  frequency  (e.g.,  Nyquist  plot). 

2.  A  system  designed  for  optimal  steady-state  operation  may  have  unstable 
transient  characteristics. 

3.  Self-organizing  adaptive  systems,  i.e.,  contrd  systems  that  incorporate 
time-vaiiance  based  on  system  operation  into  the  adaptive  scheme,  must 
have  some  means  of  evaluating  how  well  the  control  operations  are  being 
performed.  This  index  of  performance  must  be  reliable  and  unambiguous 
with  respect  to  the  optimal  range  of  operation. 

4.  It  should  be  possible  to  obtain  a  performance  index  without  disturbing 
thi*  operation  of  the  system  and  in  a  foim  w'hich  is  amenable  to  insertion 
into  that  part  of  tiie  system  in  winch  control  of  adaptation  is  accomplished. 

5.  If  hill-climbing  techniques  are  used  to  control  steady-state  adaptation 
[e.g.,  883] ,  false  peaks  must  be  defined  and  avoided. 

Other  Mathematical  Techniques 

The  two  theories  that  follow  are  inuoduced  because  of  their  promise  tor 
advancing  our  ability  to  analyze  complex  adaptive  networks.  Their  mention 
is  abbreviated  because  of  their  novelty  and  because  so  little  is  known  at 
present  about  their  proper  application. 

Ergodic  Theory 

Ergodic  theory  ‘*is  concerned  with  the  average  behavior  of  large  collections  of 
molecules  that  move  randomly  for  indefinite  periods  of  time  . . .  Ergodic  the¬ 
orists  commonly  deal  with  measure  and  probability  spaces  and  have  developed 
powerful  theorems  involving  ramification  of  these  ideas  [S37] .”  The  reader  is 
referred  to  Kolata  [537]  for  further  discussion  of  ergodic  theory. 

Field  Theory 

"Field  theory,  as  elaborated  by  Weiss,  Wolpert,  and  others,"  indicates  that  a 
field  "can  be  defined  operationally  as  a  domain  within  which  changes  in  the 
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presumptive  fates  of  ceils  can  occur”  [300] .  Cells  may  be  assigned  positional 
values  according  to  their  physical  locations  in  the  coordinate  system  of  a  par¬ 
ticular  Held.  In  terms  of  positional  information  theory,  the  field  can  be  de¬ 
fined  as  a  set  of  cells  which  have  their  positions  specified  with  respect  to  the 
same  coordinate  system.  Further  information  is  available  elsewhere  [300]. 

Spedflc  Theories  of  Line  Labeled  Information  Handling 

A  Geometry  of  Perception-rProcessing  of  Sensory  Labeled  Information 

bfinsky  and  Papert  [671]  have  uncovered  the  beginnings  of  a  powerful 
mathematical  theory  concerning  a  geometry  of  perception  peruining  to  the 
processing  of  sensory  labeled  inforaiation.  The  theory  also  deals  with  adap¬ 
tive  features  of  the  processing.  The  topolo^c  transformation  of  problems  of 
image  recognition  and  perception  into  problems  of  line  labeled  geometry  is 
insightful  and  potentially  more  useful  than  these  authors  may  have  imagined 
originally. 

As  shown  in  Fig.  7.2S,  image  processing  involves  sets  of  receptive  elements 
that  receive  and  process  aggregates  of  sensory  labeled  information.  Each 
unique,  sensory  labeled  set  independently  processes  its  sensory  aggregate  ac¬ 
cording  to  some  function,  The  results  of  processing  by  each-$M  are  com¬ 
bined  by  meaiu  of  a  function  to  obtain  the  value,  The  problems  to  be 
resolved  are: 

1.  How  can  atnys  of  this  sort  be  organized  to  permit  a  particular  v{X)  to  be 

a  useful  designator  of  a  particular  input,  X.  at  the  receptor  elements? 

2.  Can  a  geometry  be  devised  that  will  describe  this  process  precisely  and  de¬ 
fine  some  reasonably  optimal  approads  to  this  problem? 

Minsky  and  Papert  begin  their  solution  of  these  problenu  by  pointing  out 
that  some  meaningful  restrictions  must  be  placed  on  the  function  and  the 
set  4  of  functions  iPn  if  the  geometry  is  to  be  useful.  And  they 

point  out  that  previous  treatments  of  this  type  have  been  more  anecdotal 
than  mathematical. 

It  is  also  desirable  to  introduce  variable  weighting  or  some  other  potential 
means  of  adaptation,  into  the  analysis.  As  shown  in  Fig.  726,  weiglitingsoi , 
O) , . .  .,a„  may  be  assigned  each  function  ,  s?: , . . 

In  addition,  n  may  be  replaced  by  a  summation  or  integration  funcson, 
Z,  and  a  threshold  detector,  9,  may  be  added  to  designate  a  panicuiar  value 
or  region  of  When  a  is  variable,  this  consututesa  simple  perceptron,  named 
after  the  automau  of  this  general  type  that  were  designed  by  Rosenblatt 
[838-841].  It  is  noted  by  Minsky  and  Papert  that  in  such  automata  a  tends 
to  grow  faster  than  f2  in  adaptive  processing  operations  requiring  memory 
storage. 

The  more  complex  perceptron  admits  multiple,  redundant  inputs  as  shown 
in  Fig.  727.  This  type  of  processing  of  sensory  labeled  information  corre¬ 
sponds  closely  to  that  carried  out  by  the  nervous  system  and  is  amenable  to 
analysis  by  means  of  the  Group  Invariance  Theorem. 
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A  Simple  Inuce^Processing  Automaton 


F(f.  7,2 J.  An  example  of  simple,  multiple  channel,  imafe  processing.  See 
text  for  further  details  (From  Minsky  and  Papert  {6  72].) 


Grmtp  Invarianee  Theorem 

The  Group  Invariance  Theorem  of  Minsky  and  Papert  permits  analysis  of 
perceptron  operations  (i^.,  the  geometry  of  sensory  image  processing)  by 
algebra  instead  of  statistics.  This  theorem  examines  the  relationship  between 
all  possible  receptor  activations  (all  sets  of  sensory  labels,  rj ,  rj , . .  and 
theif  representation  aaoss  a  theoretical  space  of  ot^{X)  for 

A  Ptrcrpirpii 


Fig.  7.26.  Elementary  perceptron  fa  can  be  varied).  (From  .Minsky  and  Pap¬ 
ert  [6721.) 
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Equivalence  Between  Parallel  Processing  and  Group  lavariance  Theorem 


Fig.  7,27.  A  perceprron  reduced  to  Group  Invariance  Theorem  coefficients. 
^From  Minsky  and  Papert  [672}.) 

In  effect^  the  Croup  Invariance  Theorem  permits  an  algebraic  anafysis  of 
at!  geomefnies  of  rearrangements  (or  representations)  of  the  origij:al  set  of  pos¬ 
sible  receptor  labeled  activations.  It  allows  determination  of  which  aggregates 
of  (or  values  of  tjf)  reflect  a  unique  transformation  of  the  group  of 

passible  transformations  of  the  space  of  the  receptor  labelings,  Ti  ,  r2 , . . 
upon  the  predicates,  <^1 , 

Given  any  predicate  and  group  element  g,*  Minsky  and  Papert  define 
ipg  to  be  the  predicate  that,  for  ea^  ,  has  the  value  c(X ).  Thus,  one  will 
always  have  pg(X)  *  p{gX).  4>  will  be  said  to  be  closed  under  C  if  for  every 
^  in  ^  and  g  in  G  the  predicate  pg  is  also  in  <&.  If  a  percepacn  predicate  is 
invaxiant  under  a  group,  G,  then  its  coefficients  need  depend  only  on  the 
G-«quivaience  classes  of  their  p's  (6721 . 

The  Group  Invariance  Theorem  states  that  if: 

(i)  G  is  a  finite  group  of  transformations  of  a  finite  space,  R\ 

♦Given  a  poup  G.  two  Hsures.  M  and  iV,  are  G-equivaient  if  there  is  a  member  r  of  G 
for  which  M 
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(ii)  is  a  set  of  predicates  on  chat  space  closed  under  G ; 

(ui)  ^  is  In  L  (<^)  and  invariant  under  G,  Then,  there  exists  a  linear  repre¬ 
sentation  of 

for  which  the  coefficients  depend  only  on  the  (7-equivalence  class  of 
that  is  if  g  (p'  then  j3^  =  3^'. 

£  4  is  the  set  of  ail  predicates  for  which  is  a  linear  threshold  function 
with  respect  to  and  a  predicate  is  a  function  that  has  two  possible  values, 
Le^  a  binary  function,  s!'  is  a  linear  threshold  function  with  respect  to 
is  in  I  (4^),  if  there  exists  a  number  d,  and  a  sec  of  nuenben,  one  for 
each  ^  in  <^,  such  that: 

!;'(X)-[^a^^X)>91  .  •  (7J4) 

\l/€^ 

Restrkwns  on  Perceptron  Opemtions  and  Limitations  in  Geometric  Patterns 
That  Can  Be  Recognized 

Perceptions  are  not  widiout  restrictions  in  the  types  of  operations  that  can  be 
performed  and  the  geometric  patterns  that  can  be  recognized.  • 

• 

Restrictions  of  Geometry,  The  perceptron  operations  discussed  by  Minsky 
and  Papert  have  a  receptor  geometry  restriaed  as  follows: 

1.  The  number  of  points  (or  receptive  elements)  is  limited.  Hence,  the 
predicates  of  the  points  are  of  limited  order. 

2.  The  distances  between  points  are  restricted.  Hence,  their  predicates  are 
diameter-iimited. 

Order  has  to  do  with  the  number  of  characteristic  variables  needed  to 
represent  a  set  of  particular  functions.  For  example,  the  order  of  is  the 
smallest  number,  /T,  for  which  a  set  of  precicates  can  be  found  saosfying: 

|S(^)I  <K  foraO  in  <&, 

where  5(^)  is  that  subset  of  receptors,  rj ,  rj,  . . .,  upon  which 
(the  set  of  functions  required  for  recognizing  A‘)  really  depends,  and 
1(4)  is  the  linear  threshold  function  of  d>.  the  set  of  all  preoicates  that  can 
be  defined  by  £q.  7.54. 

Linear  threshold  function  perceptron  operations  are  of  order  1.  So  are  all 
the  Boolean  functions  of  two  variables  except  for: 

L  Exclusivc-or  {XT'  ^X'Y>  0)  and 

ii.  Its  complement  identity,  X^Y  {XY  ^X*Y*  >  0) 

which  are  of  order  2. 
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Type  of  Processing  Operations,  Perceptions  are  particularly  good  at  doing 
processing  operations  of  the  types  caUed  “local**  or  “conjuncuvciy  local**  by 
Minsky  and  Papert  [672] .  By  local  is  meant  that  ail  tests  (analytic  or  logical) 
can  be  done  independently  and  the  final  decision  can  be  made  by  a  logically 
simple  procedure  such  as  unanimity  of  all  tests. 

A  predicate,  is  conjunctively  local  of  order  K  if  it  can  be  computed 
by  a  set  4  of  predicates  ^  such  that: 

L  Each  ^  depends  on  no  more  Chan  AT  points  of  the  space  R; 
a.  i/(X)  »  1 1  if  \p(X)  *  1  for  every  ^  in  <& 

\o  otherwise. 

Such  processing  will  enable  a  perception  to  distinguish  convex  from  non- 
convex  figures  at  the  receptors  by  the  test  that  if  there  exist  three  receptor 
points,  pf  and  r,  such  that  ^  is  in  the  line  segment  joining  p  and  r,  and 

p  is  in  JC 
q  is  not  in  X, 
r  is  in  X,  r 

then  the  set  X  is  not  convex  (Fig.  7.2S).  Thus,  ^convexW  is  conjunctively 
local  of  order  3  by  application  of  this  thrce-poir.*  rule  [672j . 

Interestingly,  the  determination  of  connectedness  between  points  can  be 
shown  not  to  be  conjunctively  loc^  of  any  order  in  a  diametcr-iimitsd  per- 
ceptron  processing  operation.  Hence,  pcrccptrons  of  ’diis  type  cannot  com¬ 
pute  connectedness  of  geometric  figures  whereas  they  can  compute  convexity. 
However,  as  inspection  of  Fig.  6.6C  wiilindicate,  we,  too,  have  our  difficulties 
in  deCermining  connectedness. 

Types  of  Perceptrons 

Given  that  “a  Perceptron  is  a  device  capable  of  computing  all  predicates 
which  are  linear  in  some  given  set  of  partial  predicates”  (672) ,  five  dif¬ 
ferent  types  of  perceptrons  can  be  distinguished.  They  are: 

1.  DiameteHimited  Pereeptrons^the  set  of  points  upon  which  each  c 
depends  (for  each  ip  in  some  given  set  <fr)  is  restricted  not  to  exceed  a  certain 
fixed  diameter  in  the  plane. 

2.  Order^estricted  Perceptrons--^  perceptron  has  order  </i  if  no  mem¬ 
ber  of  4  depends  on  more  than  n  points. 

3.  Gamba  Perceptrons^ezch  member  of  <P  may  depend  on  all  the  points 
but  must  be  a  linear  threshold  function,  with  each  member  of  iuelf 
being  computed  by  a  perceptron  of  order  1.  Thus, 

/ 

(each  ^1*  is  a  threshold  perceptron  of  order  1 )  and 
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Not  Convex 


Convex 


{Tint  ^ 


Fif*  7.28,  Determination  of  convexity  by  three-point  rule  [672J.  Draw  a 
ztraiiht  line  connecting  mfface  pointz  such  as  ?  and  R.  If  a  third  point,  taken 
anywhere  along  this  line,  is  inside  the  space  of  the  object,  the  surface  is 
convex. 


'J'gamba  “  f  Z  ^  >«.  ]>«  1  (7.55) 

i  i 

The  Gaxnba  pcrcepiron  is  thus  a  two-layered  perceptron.  Note,  however^ 
that  no  improvement  is  afforded  by  any  multi-layered  system,  without 
loops,  in  which  there  is  an  order  restriction  at  each  layer  wherein  only 
predicates  of  finite  order  are  computed. 

4.  Random  Perceprro/ir^the  \p'%  are  random  Boolean  functions.  They 
are  order^restneted  and  is  generated  by  a  stochastic  process  according 
to  an  assigned  distribution  function  (cf.  Rosenblatt  [  838-B4 1  ] }. 

5.  Bounded  Perceptrons-^  contains  an  infinite  number  of  «^’s,  but  all 
the  lie  in  a  finite  set  of  numbers  [6721. 

Size,  Speed,  and  Layer-Hienxrchy  Considerations  in  Perceptron  Operations 
Given  application  of  the  group  invariance  theorem  to  analysis  of  perceptrons 
of  the  above  types,  several  observations  may  be  drawn  concerning  effects  of 
size,  speed,  and  layer  or  hierarchy  of  operation. 

First,  using  more  ^memory’*  docs  not  seem  to  advance  the  kinds  or  effici¬ 
encies  of  linear  threshold  operations  that  are  performed.  This  is  interesting 
because  many  believe  that  adding  memory  will  greatly  improve  the  types  of 
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operations  that  can  be  performed.  Miasky  and  Papert  would  suggest  that 
design  is  more  important  than  size.* 

Second,  it  should  be  possible  to  ^ediy  conneciionHTiatrices  between  ele* 
ments  that  will  optimize  the  efficiency  of  processing  vis-a-vis  the  number  of 
elements  involved.  Examples  of  different  coiuiection  matrices  are  shown  in 
Fifr  7.29. 

Multilayer  Pmeptrons  w/r/r  Loops 

According  to  Minsky  and  Papert,  the  ^oup  invariance  theorem  cannot  be 
applied  to  multilayered  perceptrons  with  loops.'^  The  addition  of  loops  thus 
reopens  analytic  questions.  It  remains  to  be  seen  how  the  addition  of  loops 
limits  genera]  theories  of  sensory  information  processing  by  perceptron-like 
automata.*  Some  analytic  questions  can  be  answered  a  priori.  For  example, 
the  use  of  loops  in  processing  will  not  improve  the  speed  of  computation 
afforded  by  loop-free  serial  processing.  Other  questions  cannot.  Thus,  it  is 
undear  whether  or  not  loops  afford  the  possibility  of  more  complex  analytic 
operations.  Given  finite  order  processing,  a  prerequisite  for  mathematical 
analysis,  it  is  questionable  whether  loops  afford  any  order-improvement 
beyond  that  possible  with  a  hierarchical  multilayered  construction. 

What  loops  do  offer  is  the  possibility  of  using  the  simple  feedback  princi¬ 
ple  for  ^training''  or  error  correction.  Minsky  ana  r^^pert  believe  that  the  per¬ 
ception  convergence  theorem  provides  analytic  proof  that  where  such  “learn¬ 
ing,  adaptation  or  self  organization  does  occur,  its  occurrence  can  be  thorough¬ 
ly  elucidated  (mathematically)”  [672] . 

A  Geometry  of  Sorting-Treatment  of  Motor  Labeled  Effectuation,  Synthesis, 
and  Decision  Making 

Comparison  of  Fig.  730  with  Fig.  3.42  will  disdose  how  motor4abded  ef¬ 
fectuation  or  decision  making  is  implicit  in  the  design  of  perceptrons. 

What  has  not  been  treated  explicitly  in  the  course  of  analysis  of  peroep- 
tron  operations  is  the  geometry  of  sorting,  i.e.,  an  algebraic  analysis  of  motor 
labeled  effectuation  comparable  to  that  for  sensory  reception  presented  earlier. 
Three  positions  are  possible.  One  is  that  this  geometry  is  completely  implicit 
in  the  dassiflcation  algorithms  described  by  Minsk>*  and  Papert  (perhaps  as  a 
substructure  of  predicates).  The  second  is  that  significant  extensions  of  their 
algorithms  and  theory  need  to  be  made— perhaps  by  an  expanded  treatment  of 
conditional  probabilities  and  Markov  processes.^  The  third  position,  that  such 

*Tt  is  not  yet  dear  if  artifldal  inteUifcnce  paformed  by  a  large,  specifically  desipied 
eomputei  (capable  of  ^lopcal'*  operation)  caa  adequately  simulate  intelligence  based  on 
unincorporated  desicn  features.  Lode  may  be  used  to  approximate  the  needed  features, 
but  the  results  may  be  unsatisfactory  and  the  enors  difficult  to  detect,  as  in  some  of  the 
phenomena  illustrated  in  Chapter  6. 

■Although  it  wiU:be  recalled  that  dosed  flowpaphs,  consisting  of  loop  circuits,  can  be 
MWad. 

some  loops  can  be  eliminated  by  use  of  flowgraphs. 
k Another  class  of  algorithm  that  can  compuxe  connectedness  may  be  required-Turing 
machines  can  compute  connectedness;  perceptions  cannot. 
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Q>nnection-Matriccs 


Fig.  7.29.  Difftrtnt  connection  matrices.  Are  those  m  (A)  equivalent  to 
those  darkenetf  in  (B)?  (There  is  feedback  in  A./  Are  some  elements  and  con* 
nections  in  B  superfluous?  (Even  if  different  transfer  functions  of  several  ele* 
ments  could  be  combined,  she  connections  would  allow  unique  ^epen^e/icies 
between  mpurr.  elements,  and  outputs).  (Sketches  after  Minsky  and  Papers 
16721.) 


A  Learning  Machine 


Fig.  7.30.  A  multilayer  perceptron  capable  of  making  decisions  (d).  (Ffom 
Minsky  and  Papers  f672J.)  The  aaaptations  controlling  A  Wf^  and  A  mtfhr 
benefit  from  feedback  of  information  concerning  the  system  state. 
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a  geometry  and  analysis  is  unrealizable,  may  be  dismissed  if  one  accepts 
Minsky  and  Papert’s  view  that  workable  systems  are  subject  to  analysis  and 
this  author's  assertion  that  such  systems  are  visible  within  the  refiex  pathways 
of  the  nervous  system. 

Gasiification  Algorithms 

The  following  classification  algorithms  for  separatingdifferentsensorylabeled 
aggregates  at  d  in  Fig.  7  JO  have  been  suggested  by  Minsky  and  Papert  [672]. 


1 .  Perceptron  convergence  theorem.  Let  F  be  a  set  of  unit^ength 

vectors.  Let  ^4  *  ^  be  the  vector  notation  of  S  or  If  there  exists  a 

unit  vector  A •  and  a  number  5  >  0  such  that  >  5  for  all  in  F, 

then  a  simple  program  (see  Minsky  and  Papert  [672],  p.  167)  can  be  de¬ 
vised  that  will  converge  in  a  finite  number  of  iterations  on  a  separation  of 
all  4  e  F.  A  variation  of  this  program  (see  [672])  will  separate  more  than 
two  classes  of  input  figures:  F^ ,  Fj , .  I  .,  F^. 

A  limitation  of  this  classiGcation  ^gorithm  is  that  only  linear  sepa¬ 
rations  are  performed  optimally  by  this  method. 

2.  Bayes' linear  statistical  procedure.  Again,  let  F  be  a  set  of  unit- 
length  vectors,  with  one  vector,  ^4^*  such  that  A^  *  <&  >  5  for  all  4>  in  F.  If 


^«7 

where  cj,- . «  Lojt  (-p^) 
i  r,y 


and  P  is  the  probability  that  ,  given  that  is  in  F^.,  then  €  F  will  be 
separated  with  the  lowest  possible  error  rate,  given  that  the  s^*s  are  statisti¬ 
cally  independent.  (This  is,  remarkably,  a  linear  formula  that  can  perform 
non-linear  separation.) 

3.  Best  planes  procetfu/e— This  is  essentially  an  error-minimizing  track¬ 
ing  procedure  whereby  the  set  of  A*s  is  used  for  which  choice  of  the 
largest  A^  *  4>  gives  the  fewest  errors.  The  presence  of  false  peaks  in  hill* 
climbing  searches  by  this  method  may  limit  its  applicability. 

4.  Ciusrer  c/iai>ris— Techniques  are  used  to  minimize  the  least  square 
distance  between  different  points  in  the  receptor  array  (F)  reflected  by 
the  different  ^4^-  *  In  effect,  separation  is  performed  on  the  basis  of 
spatial  dustering  of  each  sensory  aggregate.  A  more  complete  description 
of  this  approach  ar.d  a  cluster-analysis  convergence  theorem,  vi*ith  proof, 
can  be  found  in  Miniiky  and  Papert’s  book  [6721 . 

5.  Exact  matching  or  best  mat ching^This  approach  requires  a  large 
memory  and  is  cumbersome.  Each  that  has  ever  been  encountered, 
together  with  the  identity*  of  its  associated  F-class,  is  stored.  New  inputs 
axe  *^recognized**  on  the  basis  of  match  against  the  store  contents.  With 
exact  matching,  a  tedious  .»earch  results  in  a  solution  with  no  errors.  With 


^denotes  unic  vector 
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^best’*  matching,  a  completely  different  type  of  procedure  (e.g.,  algorithms 
such  as  those  incorporating  matched  illtering-see  Woody  [1103])  is 
used  to  optimize  signal  detection,  minimize  errors,  and  reduce  search  time 
(see  1672,704,11231). 

Probability  osaDescriptor  of  Motor  Effectuation:  Tlie  Conditional  Probabiliry 
of  Sorting.  An  Algebra  of  Events 

Just  as  entropy  is  reiatablc  to  the  uncertainty  of  configurations  of  gas  mole¬ 
cules  in  a  dimensional  space,  and  provides  some  measure  thereof,  so  does 
probability  provide  a  measure  or  index  of  the  likelihood  of  events.  As  we 
have  seen  from  the  work  of  Boltzmann  and  of  Shannon,  the  events  may  be 
physical<hemical  or  they  may  be  informational-probabilistic. 

Just  as  chemical  events  may  be  described  as  occupying  a  space  [328] ,  so 
rruy  other  probabilistic e\mts  be  described  in  terms  of  the  space  they  occupy. 
The  space  of  probabilistic  events  is  described  by  set  theory  and  Venn  dia¬ 
grams  thereof.  The  sample  space  (Fig.  731)  represents  the  number  of  possible 
different  arrangements  of  sample  points  or  outcomes,  and  each  event  or  spe¬ 
cific  outcome  in  the  sample  space  can  be  assigned  a  probability  of  occurrence. 

Set  theory  is  described  by  a  set  of  axioms  that  fully  define  the  algebra  of 
events  [cf.  240] .  With  cespea  to  Fig.  7.31,  they  are: 

I.i4+S*B+^  (commuutive  law);  also  for  multiplication,  AB  »  BA 

2.  .4  +  (5  +  C)^(A^3}  +  C (associative  law);  also  for  multipIicaiion,/4(^J0  » 
{AB)Y 

3.  ^  +  Q  *  AB  +  /IC  (distributiye  law) 

4.  (A  y  *A(’*  “not”  or  die  complement  of  whatever  it  foDows) 

5.  iAB)  •A'+B‘ 

6.  AA'  *  (<fr  ■  complement  of  IT) 

1.  AU  *  A  {U  ^  union  of  two  events— the  collection  of  all  points  in  either  or 
both  event  spaces) 

Thix  S€t  of  axioms  is  also  the  set  of  constraints  by  which  linear  systems  are 
bound  and  defined. 

Simple  Probability.  Could  probability  be  used  to  describe  motor  effectu* 
ation,  Le.,  the  motor  events  (or  decisional  space)  possible  as  outcomes  of  a 
particular  network?  If  so,  could  some  general  fonnulaiion  be  derived,  com¬ 
parable  to  the  group  invariance  theorem  to  permit  a  general  algebraic  treat¬ 
ment  of  the  geometry  of  sorting  or  motor  effectuation?  The  answer  to  the 
first  question  is  yes;  the  answer  to  the  second,  perhaps.  The  sample  space, 
of  possible  motor  outcomes  is  made  up  of  a  number  of  points,  £| 

Each  point,  has  an  expected  probability  of  occurrence  £(£^).* 
The  probability  of  occurrence  of  event  A.  P(A),  is  the  sum  of  the  proba¬ 
bilities  of  all  points  within  it.  The  sum  of  the  probabilities  of  occurrence  of 
all  points  equals  1,  which  is  equal  to  the  probability  of  the  entire  sample 
space.  Thus, P(A)  must  be  between  0  and  1. 

‘Event  A  m»y  be  mapped  from  sets  otPfEiJ. 
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Fig.  7,32,  The  sample  space  of  K  B.  Qand  f  A^B.Q . 

Conditional  Probability,  Conditional  probability  deals  with  the  probability 
of  an  event  A  occurring  given  that  some  other  event  B  has  just  occurred.  If 
the  events  are  completely  independent,  the  probability  of  events  occurring 
will  be  equal  to  the  general  probabilizy  of  occurrence  of  event  >4,  ^(.4).  If 
there  is  some  dependency,  the  prcbaoilicy  of  event  A  occurring,  once  B  has 
occurred,  may  be  different  from  the  general  probability  of  occurrence  of 
event  A,  Bayes  has  systematized  this  relationship.  If  one  thinks  of  B  as  the 
causal  event  and  i4  as  the  affected  event,  the  probability  that  A  occurs  given 
that  B  has  occurred,  is  equal  to  the  general  probability  of  occurrence 

of  times  the  probability  of  the  effect  B  given  that  the  phenomenon 

A  has  occurred,  ^(B/i4),  divided  by  the  probability  of  event  3^P{B),  Thus: 


PiMB) 


nA)P{BlA) 

P{.B) 


(7-S6) 


Interestingly,  this  theorem  may  be  generalized  to  encompass  the  relation* 
ship  of  a  set  of  events  Ai,  A^,  . . Aff.  This  is  because  P{B)  will  equal 
P[(4i  +^i  +  . .  >A,^\  or 


P{B)^P{.AiB) 


(7-57) 


It  can  be  shown  that: 


yv  iT 

'^PiAiB)  ^P{Ai)P{BlAi). 


********** 


(7.58) 
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University  of  California,  Los  Angeles 
Campus  Vcterina.rian,  l\?-21 1  C.H.S. 
Telephone:  (2131 825-G240 


_ Office  Use  Only _ 

ApplicatiM  U  3  ^  (ly  in  a 

Efioctive  Psspiraiinfi _ 


1 


y-  -y  k:; 


APPLICATION  FOR  USE  OF  LABORATORY  ANIMAL  SUBJECTS 


A  separate  app)>caiton  must  be  completed  and  typewritten  for  each  proposed  project  or  activity  utilizing 
animals.*  Applications  should  be  directed  to  the  Campus  Veterinarian  for  review,  but  can  be  sent  after  funding 
agency  deadlines.  Hc-.vevcr.  an  approved  application  is  rCQuircd  for  the  University  to  accept  extramural  funds 
and  before  any  animals  can  be  ordered. 

Charles  D.  Woody,  MD 


Applicant. 


Phone:  Of f ice  825‘"Q1_87^  _ 

NPI/MR 


.Home. 


Depanmont/Oivision. 


Title  of  Research  or  Training  Projeci/Activit>> _ Wgurnphys i  oT ngf  ra  1  Rp^parr h  Support i  nq 

the  Investigation  of  Adaptive  Network  Architectures 


Estimated  Starting  flAtg  _  6/T  ^0 


.Estimated  Completion  nat»  5/31 19^ 


The  undersigned  attests  to  tlie  attached  information,  and  agrees  to  accept  responsibility  that  all  animal  use  in  the 
above*titlcd  project  or  activity  will  be  in  accordance  with  University.  Federal,  and  other  relevant  policies  and 
regulations.  Any  changes  will  be  communicated  to  the  Chancellor's  Committee/Campus  Veterinarian. 


Signature. 


>  irCZi. 


Charles  D,  Woody,  MD  ^o^esior  of  Anatomy  and  Psychiat^ry 


•**Animar’  means  sny  Uve  or  dend  verfePrafe. 


For  Committee  Use  Only 

y<, 

Comments: 


^  Effective  Date  '**  ^ ^  ^  _  r— r 

Appfication  Approved;  Expiration  Date  f  —  JT  ■<  .  ( _ |  Application  Disapproved 
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Untvtriiiy  qI  California.  Los  Angelas 
Campus  Veterinarian.  IV-211  C.H.S. 
Telephone:  (213)  825-6240 


Office  Use  Only 

Application  f 


.»r  Pfoiect  10  4 

^  "»  /•^  k'C>  guptratinn  _  j  ^ 


^6 


APPLICATION  FOR  USE  OF  LABORATORY  ANIMAL  SUBJECTS 

.  ■  Department _ -  NPf/Mft 


1.  Applicant.  Charles  D.  Woodv..  MD 

Phone:  Qflige  -Home 


2.  Contact  for  animal  problems  &  emergencies...^ 
Phone:  Office  825*01ft7  Home 


3.  Til 


Charles  D.  Wnorfy,  Mn 


itit  of  rcietreh  w  .training  moicM/activlty _  NeurQPhySTp]  Q^ltcal _ Rpsparrft  ^iippnrttnn  fhy 

Investigation  of  Adaptive  Network  Architecures  ' '  - 

4.  Starting  Data.  6/17  80  to  5/31/S^  Q  New  Q Continuation  O Renewal  O Supplemental 

5.  fundir^.  Acad.  Senate,  or  Cal.  Inst,  for  Cancer  Res.:  Artech  copy  of  propoat  Mpplie»tion. 

6.  Q  Intramural  or  other  funding:  On  cont,  popo  outUno  proioct’s  oO/cerrVer.  tnimot  purchosin^  &  care  budgoL 

7.  Animal  Usa  Sifts  —  buildinofsl/roomlsl  58*1 47  >  58*1  59  »  57*384. 

8.  For  animals  held  in  laboratory  more  than  12  hours,  does  housing  conform  to  OHEW  Guide:  Q  Yes  □  No 

*9.  Specie^^tfain/Br— d _  Cflt  Sex  Age/Weight  R^nge  -  _  _  —  - 

20-40 

10.  Total  number  for  entire  ofoiect/aetiwitv  Expected  Daily  Population  *  ^  _  _  _ _ 

11.0  Special  procurement  or  processing  needs:  Spocify  on  conr/nuar/dn  page 

12. 0  Procurement  of  dead  animal  material:  Answer  gueft/dnx  onfy  »nd  pgn  ^ppiicMtiofu 

13.  O  Short  term  use  (up  to  2  weeks!  O  ^onq  term  use  (more  than  2  weeksi  Q  Both  O  Breeding  program 

14.  O  Standard  housing,  diet,  sanitation  &  pest  control  O  Special  needs:  Sptcify  on  eontinuttion  psgo. 

15.  O  Project  involves  no  pain  or  distress  to  animal  subjects.  Re/er  to  '^Guiding  Phneiptos"  iPS, 

16. 0  Project  involves  probable  pain  or  distress  to  animal  subjectt.  Ootsils  of  procedures  on  enimets 

presented  in:  O  proposel  eppOcevon  ^conttnuetion  pege  O  attached  iournai  reprint 
17.  G9  Fain  or  distress  tranquilizers.  Use  cont,  for  ed^  drugs. 


Cite:  Drug(sl^ 


Routt. 


IP 


18.  O  Fain  or  distress  cannot  be  relieved.  Sxpiain  basis  for  exception  on  conr/nuar/on  page, 

19.  Q  Surgery:  Q  Nonsurvival  ^3  Survival  fXI  Aseptic  Surgery  Q  Multiple  surgeries  on  same  animal 

20.  Frocedurt  for  sick/daad  animals:  O  Veterinary  stttntion  Q  Discard  C9  Notify  applicant 

Routt  IP 


21.  Q  Euthanasia:  Cite:  druaUI  Nenibutal 
Cite  other  meihodfsl  - 


22.  □  Spteial  vcitrinarv  &  technical  services  required:  Spteify  on  centinuttion  page. 

23.  O  Potential  biologicil  or  radiation  hatard  to:  O  Humans  O  Animals  Describe  on  eennnuetion  page. 

use  CONTINUATION  PA0S(St  FO/i  ADDITIONAL  INFORMATION 
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